2024-06-01T01:41:41.101 INFO:root:teuthology version: 0.0.1.dev303+g861a8dc 2024-06-01T01:41:41.102 DEBUG:teuthology.run:Teuthology command: teuthology --name teuthology-2024-05-31_21:08:02-orch-squid-distro-default-smithi --verbose --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2024-05-31_21:08:02-orch-squid-distro-default-smithi/7736361 --description orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} -- /home/teuthworker/archive/teuthology-2024-05-31_21:08:02-orch-squid-distro-default-smithi/7736361/orig.config.yaml 2024-06-01T01:41:41.132 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-01T01:41:41.218 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-05-31_21:08:02-orch-squid-distro-default-smithi/7736361 branch: squid description: orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} email: ceph-qa@ceph.com first_in_suite: false job_id: '7736361' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-05-31_21:08:02-orch-squid-distro-default-smithi no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: global: mon election default strategy: 3 mgr: debug mgr: 20 debug ms: 1 mgr/cephadm/use_agent: true mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) log-only-match: - CEPHADM_ sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba workunit: branch: squid sha1: 6dca6333905c2b38b02396dfd1a76a97d29c2ee7 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b seed: 3011 sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba sleep_before_teardown: 0 subset: 21/64 suite: orch suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_6dca6333905c2b38b02396dfd1a76a97d29c2ee7/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 6dca6333905c2b38b02396dfd1a76a97d29c2ee7 targets: smithi116.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI0adaeJQDEWrg3rk2iH4GyTFs0ZrAdAwfzbG1KCjLXx40shIaGliPZEAy1B2m+cquiLtvgEilMT3je7mJAEZ+U= smithi161.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBANZU9EgYgKn1oZNwO5tBuhfM/E8q+GtzjCg/CRCsQv6zqssMJZE0aFYXyiOGitxrDCFLRon+fuFRwxlq+T/lKs= tasks: - cephadm: cephadm_branch: v17.2.0 cephadm_git_url: https://github.com/ceph/ceph image: quay.io/ceph/ceph:v17.2.0 - cephadm.shell: mon.a: - ceph config set mgr mgr/cephadm/use_repo_digest false --force - cephadm.shell: env: - sha1 mon.a: - radosgw-admin realm create --rgw-realm=r --default - radosgw-admin zonegroup create --rgw-zonegroup=default --master --default - radosgw-admin zone create --rgw-zonegroup=default --rgw-zone=z --master --default - radosgw-admin period update --rgw-realm=r --commit - ceph orch apply rgw foo --realm r --zone z --placement=2 --port=8000 - ceph orch apply rgw smpl - ceph osd pool create foo - rbd pool init foo - ceph orch apply iscsi foo u p - sleep 120 - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 mon.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph orch upgrade status - ceph health detail - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 - ceph orch ls | grep '^osd ' - cephadm.shell: mon.a: - ceph orch upgrade ls - ceph orch upgrade ls --image quay.io/ceph/ceph --show-all-versions | grep 16.2.0 - ceph orch upgrade ls --image quay.io/ceph/ceph --tags | grep v16.2.2 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 861a8dcf7aa816a26e13f039336f7ed0a9aec0fa timestamp: 2024-05-31_21:08:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-06-01T01:41:41.218 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_6dca6333905c2b38b02396dfd1a76a97d29c2ee7/qa; will attempt to use it 2024-06-01T01:41:41.219 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_6dca6333905c2b38b02396dfd1a76a97d29c2ee7/qa/tasks 2024-06-01T01:41:41.219 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-06-01T01:41:41.220 INFO:teuthology.task.internal:Checking packages... 2024-06-01T01:41:41.240 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3d89accfe3201286f000e065a1faea2a88c7a6ba' 2024-06-01T01:41:41.241 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-06-01T01:41:41.241 INFO:teuthology.packaging:ref: None 2024-06-01T01:41:41.241 INFO:teuthology.packaging:tag: None 2024-06-01T01:41:41.241 INFO:teuthology.packaging:branch: squid 2024-06-01T01:41:41.241 INFO:teuthology.packaging:sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T01:41:41.241 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-06-01T01:41:41.452 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2946.g3d89accf 2024-06-01T01:41:41.453 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-06-01T01:41:41.460 INFO:teuthology.task.internal:no buildpackages task found 2024-06-01T01:41:41.460 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-06-01T01:41:41.468 INFO:teuthology.task.internal:Saving configuration 2024-06-01T01:41:41.479 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-06-01T01:41:41.486 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-06-01T01:41:41.511 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi116.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-31_21:08:02-orch-squid-distro-default-smithi/7736361', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-01 01:36:41.573574', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI0adaeJQDEWrg3rk2iH4GyTFs0ZrAdAwfzbG1KCjLXx40shIaGliPZEAy1B2m+cquiLtvgEilMT3je7mJAEZ+U='} 2024-06-01T01:41:41.528 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-31_21:08:02-orch-squid-distro-default-smithi/7736361', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-01 01:36:41.574859', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBANZU9EgYgKn1oZNwO5tBuhfM/E8q+GtzjCg/CRCsQv6zqssMJZE0aFYXyiOGitxrDCFLRon+fuFRwxlq+T/lKs='} 2024-06-01T01:41:41.528 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-06-01T01:41:41.534 INFO:teuthology.task.internal:roles: ubuntu@smithi116.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a'] 2024-06-01T01:41:41.534 INFO:teuthology.task.internal:roles: ubuntu@smithi161.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b'] 2024-06-01T01:41:41.535 INFO:teuthology.run_tasks:Running task console_log... 2024-06-01T01:41:41.588 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe0c5643ac0>, signals=[15]) 2024-06-01T01:41:41.588 INFO:teuthology.run_tasks:Running task internal.connect... 2024-06-01T01:41:41.594 INFO:teuthology.task.internal:Opening connections... 2024-06-01T01:41:41.594 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi116.front.sepia.ceph.com 2024-06-01T01:41:41.595 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-01T01:41:41.669 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi161.front.sepia.ceph.com 2024-06-01T01:41:41.670 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-01T01:41:41.748 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-06-01T01:41:41.755 DEBUG:teuthology.orchestra.run.smithi116:> uname -m 2024-06-01T01:41:41.774 INFO:teuthology.orchestra.run.smithi116.stdout:x86_64 2024-06-01T01:41:41.774 DEBUG:teuthology.orchestra.run.smithi116:> cat /etc/os-release 2024-06-01T01:41:41.830 INFO:teuthology.orchestra.run.smithi116.stdout:NAME="CentOS Stream" 2024-06-01T01:41:41.831 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION="9" 2024-06-01T01:41:41.831 INFO:teuthology.orchestra.run.smithi116.stdout:ID="centos" 2024-06-01T01:41:41.831 INFO:teuthology.orchestra.run.smithi116.stdout:ID_LIKE="rhel fedora" 2024-06-01T01:41:41.831 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION_ID="9" 2024-06-01T01:41:41.831 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM_ID="platform:el9" 2024-06-01T01:41:41.831 INFO:teuthology.orchestra.run.smithi116.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-01T01:41:41.831 INFO:teuthology.orchestra.run.smithi116.stdout:ANSI_COLOR="0;31" 2024-06-01T01:41:41.831 INFO:teuthology.orchestra.run.smithi116.stdout:LOGO="fedora-logo-icon" 2024-06-01T01:41:41.831 INFO:teuthology.orchestra.run.smithi116.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-01T01:41:41.831 INFO:teuthology.orchestra.run.smithi116.stdout:HOME_URL="https://centos.org/" 2024-06-01T01:41:41.831 INFO:teuthology.orchestra.run.smithi116.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-01T01:41:41.831 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-01T01:41:41.831 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-01T01:41:41.832 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server 2024-06-01T01:41:41.851 DEBUG:teuthology.orchestra.run.smithi161:> uname -m 2024-06-01T01:41:41.868 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64 2024-06-01T01:41:41.869 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release 2024-06-01T01:41:41.926 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="CentOS Stream" 2024-06-01T01:41:41.926 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="9" 2024-06-01T01:41:41.926 INFO:teuthology.orchestra.run.smithi161.stdout:ID="centos" 2024-06-01T01:41:41.926 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="rhel fedora" 2024-06-01T01:41:41.927 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="9" 2024-06-01T01:41:41.927 INFO:teuthology.orchestra.run.smithi161.stdout:PLATFORM_ID="platform:el9" 2024-06-01T01:41:41.927 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-01T01:41:41.927 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31" 2024-06-01T01:41:41.927 INFO:teuthology.orchestra.run.smithi161.stdout:LOGO="fedora-logo-icon" 2024-06-01T01:41:41.927 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-01T01:41:41.927 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://centos.org/" 2024-06-01T01:41:41.927 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-01T01:41:41.927 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-01T01:41:41.927 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-01T01:41:41.927 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2024-06-01T01:41:41.951 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-06-01T01:41:41.961 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-06-01T01:41:41.967 INFO:teuthology.task.internal:Checking for old test directory... 2024-06-01T01:41:41.967 DEBUG:teuthology.orchestra.run.smithi116:> test '!' -e /home/ubuntu/cephtest 2024-06-01T01:41:41.969 DEBUG:teuthology.orchestra.run.smithi161:> test '!' -e /home/ubuntu/cephtest 2024-06-01T01:41:41.986 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-06-01T01:41:41.992 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-06-01T01:41:41.993 DEBUG:teuthology.orchestra.run.smithi116:> test -z $(ls -A /var/lib/ceph) 2024-06-01T01:41:42.026 DEBUG:teuthology.orchestra.run.smithi161:> test -z $(ls -A /var/lib/ceph) 2024-06-01T01:41:42.062 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-06-01T01:41:42.119 INFO:teuthology.run_tasks:Running task kernel... 2024-06-01T01:41:42.133 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-06-01T01:41:42.133 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-06-01T01:41:42.133 DEBUG:teuthology.orchestra.run.smithi116:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-01T01:41:42.134 DEBUG:teuthology.orchestra.run.smithi161:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-01T01:41:42.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-01T01:41:42.149 DEBUG:teuthology.orchestra.run.smithi116:> uname -r 2024-06-01T01:41:42.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-01T01:41:42.151 DEBUG:teuthology.orchestra.run.smithi161:> uname -r 2024-06-01T01:41:42.206 INFO:teuthology.orchestra.run.smithi116.stdout:5.14.0-437.el9.x86_64 2024-06-01T01:41:42.206 INFO:teuthology.task.kernel:Running kernel on smithi116: 5.14.0-437.el9.x86_64 2024-06-01T01:41:42.206 DEBUG:teuthology.orchestra.run.smithi116:> sudo yum install -y kernel 2024-06-01T01:41:42.207 INFO:teuthology.orchestra.run.smithi161.stdout:5.14.0-437.el9.x86_64 2024-06-01T01:41:42.207 INFO:teuthology.task.kernel:Running kernel on smithi161: 5.14.0-437.el9.x86_64 2024-06-01T01:41:42.208 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum install -y kernel 2024-06-01T01:41:45.369 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - BaseOS 5.2 MB/s | 8.1 MB 00:01 2024-06-01T01:41:45.849 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS Stream 9 - BaseOS 3.9 MB/s | 8.1 MB 00:02 2024-06-01T01:41:47.287 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-06-01T01:41:47.558 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS Stream 9 - AppStream 31 MB/s | 19 MB 00:00 2024-06-01T01:41:52.938 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - CRB 3.1 MB/s | 6.2 MB 00:02 2024-06-01T01:41:53.025 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.2 MB 00:01 2024-06-01T01:41:54.435 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS Stream 9 - Extras packages 66 kB/s | 17 kB 00:00 2024-06-01T01:41:54.455 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 17 kB 00:00 2024-06-01T01:41:55.269 INFO:teuthology.orchestra.run.smithi116.stdout:Extra Packages for Enterprise Linux 30 MB/s | 21 MB 00:00 2024-06-01T01:41:56.056 INFO:teuthology.orchestra.run.smithi161.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-06-01T01:42:00.736 INFO:teuthology.orchestra.run.smithi116.stdout:lab-extras 5.5 kB/s | 1.7 kB 00:00 2024-06-01T01:42:01.364 INFO:teuthology.orchestra.run.smithi161.stdout:lab-extras 10 kB/s | 1.7 kB 00:00 2024-06-01T01:42:02.742 INFO:teuthology.orchestra.run.smithi116.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-06-01T01:42:02.742 INFO:teuthology.orchestra.run.smithi116.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-06-01T01:42:02.835 INFO:teuthology.orchestra.run.smithi116.stdout:Dependencies resolved. 2024-06-01T01:42:02.845 INFO:teuthology.orchestra.run.smithi116.stdout:================================================================================ 2024-06-01T01:42:02.845 INFO:teuthology.orchestra.run.smithi116.stdout: Package Architecture Version Repository Size 2024-06-01T01:42:02.845 INFO:teuthology.orchestra.run.smithi116.stdout:================================================================================ 2024-06-01T01:42:02.845 INFO:teuthology.orchestra.run.smithi116.stdout:Installing: 2024-06-01T01:42:02.845 INFO:teuthology.orchestra.run.smithi116.stdout: kernel x86_64 5.14.0-452.el9 baseos 893 k 2024-06-01T01:42:02.845 INFO:teuthology.orchestra.run.smithi116.stdout:Installing dependencies: 2024-06-01T01:42:02.845 INFO:teuthology.orchestra.run.smithi116.stdout: kernel-core x86_64 5.14.0-452.el9 baseos 16 M 2024-06-01T01:42:02.845 INFO:teuthology.orchestra.run.smithi116.stdout: kernel-modules x86_64 5.14.0-452.el9 baseos 35 M 2024-06-01T01:42:02.845 INFO:teuthology.orchestra.run.smithi116.stdout: kernel-modules-core x86_64 5.14.0-452.el9 baseos 29 M 2024-06-01T01:42:02.846 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-06-01T01:42:02.846 INFO:teuthology.orchestra.run.smithi116.stdout:Transaction Summary 2024-06-01T01:42:02.846 INFO:teuthology.orchestra.run.smithi116.stdout:================================================================================ 2024-06-01T01:42:02.846 INFO:teuthology.orchestra.run.smithi116.stdout:Install 4 Packages 2024-06-01T01:42:02.846 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-06-01T01:42:02.846 INFO:teuthology.orchestra.run.smithi116.stdout:Total download size: 80 M 2024-06-01T01:42:02.847 INFO:teuthology.orchestra.run.smithi116.stdout:Installed size: 124 M 2024-06-01T01:42:02.847 INFO:teuthology.orchestra.run.smithi116.stdout:Downloading Packages: 2024-06-01T01:42:03.320 INFO:teuthology.orchestra.run.smithi161.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-06-01T01:42:03.321 INFO:teuthology.orchestra.run.smithi161.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-06-01T01:42:03.413 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-06-01T01:42:03.422 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-06-01T01:42:03.422 INFO:teuthology.orchestra.run.smithi161.stdout: Package Architecture Version Repository Size 2024-06-01T01:42:03.423 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-06-01T01:42:03.423 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2024-06-01T01:42:03.423 INFO:teuthology.orchestra.run.smithi161.stdout: kernel x86_64 5.14.0-452.el9 baseos 893 k 2024-06-01T01:42:03.423 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2024-06-01T01:42:03.423 INFO:teuthology.orchestra.run.smithi161.stdout: kernel-core x86_64 5.14.0-452.el9 baseos 16 M 2024-06-01T01:42:03.423 INFO:teuthology.orchestra.run.smithi161.stdout: kernel-modules x86_64 5.14.0-452.el9 baseos 35 M 2024-06-01T01:42:03.423 INFO:teuthology.orchestra.run.smithi161.stdout: kernel-modules-core x86_64 5.14.0-452.el9 baseos 29 M 2024-06-01T01:42:03.423 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-06-01T01:42:03.423 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-06-01T01:42:03.423 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-06-01T01:42:03.423 INFO:teuthology.orchestra.run.smithi161.stdout:Install 4 Packages 2024-06-01T01:42:03.423 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-06-01T01:42:03.424 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 80 M 2024-06-01T01:42:03.424 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 124 M 2024-06-01T01:42:03.424 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-06-01T01:42:03.612 INFO:teuthology.orchestra.run.smithi116.stdout:(1/4): kernel-5.14.0-452.el9.x86_64.rpm 1.8 MB/s | 893 kB 00:00 2024-06-01T01:42:03.691 INFO:teuthology.orchestra.run.smithi161.stdout:(1/4): kernel-5.14.0-452.el9.x86_64.rpm 4.4 MB/s | 893 kB 00:00 2024-06-01T01:42:03.787 INFO:teuthology.orchestra.run.smithi116.stdout:(2/4): kernel-core-5.14.0-452.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-06-01T01:42:04.002 INFO:teuthology.orchestra.run.smithi161.stdout:(2/4): kernel-core-5.14.0-452.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-06-01T01:42:04.937 INFO:teuthology.orchestra.run.smithi116.stdout:(3/4): kernel-modules-5.14.0-452.el9.x86_64.rpm 19 MB/s | 35 MB 00:01 2024-06-01T01:42:04.941 INFO:teuthology.orchestra.run.smithi161.stdout:(3/4): kernel-modules-5.14.0-452.el9.x86_64.rpm 24 MB/s | 35 MB 00:01 2024-06-01T01:42:05.366 INFO:teuthology.orchestra.run.smithi161.stdout:(4/4): kernel-modules-core-5.14.0-452.el9.x86_6 17 MB/s | 29 MB 00:01 2024-06-01T01:42:05.367 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-06-01T01:42:05.367 INFO:teuthology.orchestra.run.smithi161.stdout:Total 41 MB/s | 80 MB 00:01 2024-06-01T01:42:05.371 INFO:teuthology.orchestra.run.smithi116.stdout:(4/4): kernel-modules-core-5.14.0-452.el9.x86_6 16 MB/s | 29 MB 00:01 2024-06-01T01:42:05.371 INFO:teuthology.orchestra.run.smithi116.stdout:-------------------------------------------------------------------------------- 2024-06-01T01:42:05.371 INFO:teuthology.orchestra.run.smithi116.stdout:Total 32 MB/s | 80 MB 00:02 2024-06-01T01:42:06.065 INFO:teuthology.orchestra.run.smithi116.stdout:Running transaction check 2024-06-01T01:42:06.068 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-06-01T01:42:06.408 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-06-01T01:42:06.409 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-06-01T01:42:06.414 INFO:teuthology.orchestra.run.smithi116.stdout:Transaction check succeeded. 2024-06-01T01:42:06.414 INFO:teuthology.orchestra.run.smithi116.stdout:Running transaction test 2024-06-01T01:42:07.361 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-06-01T01:42:07.362 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-06-01T01:42:07.364 INFO:teuthology.orchestra.run.smithi116.stdout:Transaction test succeeded. 2024-06-01T01:42:07.364 INFO:teuthology.orchestra.run.smithi116.stdout:Running transaction 2024-06-01T01:42:09.745 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-06-01T01:42:09.768 INFO:teuthology.orchestra.run.smithi116.stdout: Preparing : 1/1 2024-06-01T01:42:10.403 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : kernel-modules-core-5.14.0-452.el9.x86_64 1/4 2024-06-01T01:42:10.487 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : kernel-modules-core-5.14.0-452.el9.x86_64 1/4 2024-06-01T01:42:10.492 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-01T01:42:10.571 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-01T01:42:13.151 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-01T01:42:13.242 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-01T01:42:13.296 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-01T01:42:13.381 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-01T01:42:18.018 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-01T01:42:18.101 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : kernel-5.14.0-452.el9.x86_64 4/4 2024-06-01T01:42:18.116 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-01T01:42:18.386 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : kernel-5.14.0-452.el9.x86_64 4/4 2024-06-01T01:42:22.888 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-01T01:42:23.155 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-01T01:42:52.378 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 4/4 2024-06-01T01:42:52.417 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 4/4 2024-06-01T01:42:52.793 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: kernel-5.14.0-452.el9.x86_64 4/4 2024-06-01T01:42:52.794 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : kernel-5.14.0-452.el9.x86_64 1/4 2024-06-01T01:42:52.794 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-01T01:42:52.794 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-01T01:42:52.871 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 4/4 2024-06-01T01:42:52.911 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 4/4 2024-06-01T01:42:53.220 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: kernel-5.14.0-452.el9.x86_64 4/4 2024-06-01T01:42:53.221 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : kernel-5.14.0-452.el9.x86_64 1/4 2024-06-01T01:42:53.221 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-01T01:42:53.221 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-01T01:42:53.601 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-01T01:42:53.601 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-06-01T01:42:53.601 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-06-01T01:42:53.601 INFO:teuthology.orchestra.run.smithi161.stdout: kernel-5.14.0-452.el9.x86_64 2024-06-01T01:42:53.602 INFO:teuthology.orchestra.run.smithi161.stdout: kernel-core-5.14.0-452.el9.x86_64 2024-06-01T01:42:53.602 INFO:teuthology.orchestra.run.smithi161.stdout: kernel-modules-5.14.0-452.el9.x86_64 2024-06-01T01:42:53.602 INFO:teuthology.orchestra.run.smithi161.stdout: kernel-modules-core-5.14.0-452.el9.x86_64 2024-06-01T01:42:53.602 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-06-01T01:42:53.602 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-06-01T01:42:53.996 DEBUG:teuthology.orchestra.run.smithi161:> rpm -q kernel | sort -rV | head -n 1 2024-06-01T01:42:54.015 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-01T01:42:54.015 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-06-01T01:42:54.015 INFO:teuthology.orchestra.run.smithi116.stdout:Installed: 2024-06-01T01:42:54.015 INFO:teuthology.orchestra.run.smithi116.stdout: kernel-5.14.0-452.el9.x86_64 2024-06-01T01:42:54.016 INFO:teuthology.orchestra.run.smithi116.stdout: kernel-core-5.14.0-452.el9.x86_64 2024-06-01T01:42:54.016 INFO:teuthology.orchestra.run.smithi116.stdout: kernel-modules-5.14.0-452.el9.x86_64 2024-06-01T01:42:54.016 INFO:teuthology.orchestra.run.smithi116.stdout: kernel-modules-core-5.14.0-452.el9.x86_64 2024-06-01T01:42:54.016 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-06-01T01:42:54.016 INFO:teuthology.orchestra.run.smithi116.stdout:Complete! 2024-06-01T01:42:54.076 INFO:teuthology.orchestra.run.smithi161.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-01T01:42:54.076 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-01T01:42:54.076 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-452.el9.x86_64 2024-06-01T01:42:54.076 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-06-01T01:42:54.076 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-06-01T01:42:54.076 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-06-01T01:42:54.077 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi161.front.sepia.ceph.com, path=None, version=distro) 2024-06-01T01:42:54.077 DEBUG:teuthology.orchestra.run.smithi161:> rpm -q kernel | sort -rV | head -n 1 2024-06-01T01:42:54.128 INFO:teuthology.orchestra.run.smithi161.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-01T01:42:54.128 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-01T01:42:54.128 DEBUG:teuthology.orchestra.run.smithi161:> sudo rpm -qi grub2-tools 2024-06-01T01:42:54.159 INFO:teuthology.orchestra.run.smithi161.stdout:Name : grub2-tools 2024-06-01T01:42:54.159 INFO:teuthology.orchestra.run.smithi161.stdout:Epoch : 1 2024-06-01T01:42:54.159 INFO:teuthology.orchestra.run.smithi161.stdout:Version : 2.06 2024-06-01T01:42:54.159 INFO:teuthology.orchestra.run.smithi161.stdout:Release : 68.el9 2024-06-01T01:42:54.159 INFO:teuthology.orchestra.run.smithi161.stdout:Architecture: x86_64 2024-06-01T01:42:54.159 INFO:teuthology.orchestra.run.smithi161.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-01T01:42:54.159 INFO:teuthology.orchestra.run.smithi161.stdout:Group : Unspecified 2024-06-01T01:42:54.159 INFO:teuthology.orchestra.run.smithi161.stdout:Size : 8264254 2024-06-01T01:42:54.159 INFO:teuthology.orchestra.run.smithi161.stdout:License : GPLv3+ 2024-06-01T01:42:54.159 INFO:teuthology.orchestra.run.smithi161.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-01T01:42:54.160 INFO:teuthology.orchestra.run.smithi161.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-01T01:42:54.160 INFO:teuthology.orchestra.run.smithi161.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-01T01:42:54.160 INFO:teuthology.orchestra.run.smithi161.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-01T01:42:54.160 INFO:teuthology.orchestra.run.smithi161.stdout:Packager : builder@centos.org 2024-06-01T01:42:54.160 INFO:teuthology.orchestra.run.smithi161.stdout:Vendor : CentOS 2024-06-01T01:42:54.160 INFO:teuthology.orchestra.run.smithi161.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-01T01:42:54.160 INFO:teuthology.orchestra.run.smithi161.stdout:Summary : Support tools for GRUB. 2024-06-01T01:42:54.160 INFO:teuthology.orchestra.run.smithi161.stdout:Description : 2024-06-01T01:42:54.160 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-06-01T01:42:54.160 INFO:teuthology.orchestra.run.smithi161.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-01T01:42:54.160 INFO:teuthology.orchestra.run.smithi161.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-01T01:42:54.160 INFO:teuthology.orchestra.run.smithi161.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-01T01:42:54.160 INFO:teuthology.orchestra.run.smithi161.stdout:hardware devices. 2024-06-01T01:42:54.161 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-06-01T01:42:54.161 INFO:teuthology.orchestra.run.smithi161.stdout:This subpackage provides tools for support of all platforms. 2024-06-01T01:42:54.161 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-01T01:42:54.161 INFO:teuthology.task.kernel:Updating grub on smithi161 to boot 5.14.0-452.el9.x86_64 2024-06-01T01:42:54.162 DEBUG:teuthology.orchestra.run.smithi161:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-01T01:42:54.364 DEBUG:teuthology.orchestra.run.smithi116:> rpm -q kernel | sort -rV | head -n 1 2024-06-01T01:42:54.444 INFO:teuthology.orchestra.run.smithi116.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-01T01:42:54.444 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-01T01:42:54.444 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-452.el9.x86_64 2024-06-01T01:42:54.445 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-06-01T01:42:54.445 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-06-01T01:42:54.445 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-06-01T01:42:54.445 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi116.front.sepia.ceph.com, path=None, version=distro) 2024-06-01T01:42:54.445 DEBUG:teuthology.orchestra.run.smithi116:> rpm -q kernel | sort -rV | head -n 1 2024-06-01T01:42:54.497 INFO:teuthology.orchestra.run.smithi116.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-01T01:42:54.497 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-01T01:42:54.498 DEBUG:teuthology.orchestra.run.smithi116:> sudo rpm -qi grub2-tools 2024-06-01T01:42:54.532 INFO:teuthology.orchestra.run.smithi116.stdout:Name : grub2-tools 2024-06-01T01:42:54.532 INFO:teuthology.orchestra.run.smithi116.stdout:Epoch : 1 2024-06-01T01:42:54.532 INFO:teuthology.orchestra.run.smithi116.stdout:Version : 2.06 2024-06-01T01:42:54.532 INFO:teuthology.orchestra.run.smithi116.stdout:Release : 68.el9 2024-06-01T01:42:54.532 INFO:teuthology.orchestra.run.smithi116.stdout:Architecture: x86_64 2024-06-01T01:42:54.533 INFO:teuthology.orchestra.run.smithi116.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-01T01:42:54.533 INFO:teuthology.orchestra.run.smithi116.stdout:Group : Unspecified 2024-06-01T01:42:54.533 INFO:teuthology.orchestra.run.smithi116.stdout:Size : 8264254 2024-06-01T01:42:54.533 INFO:teuthology.orchestra.run.smithi116.stdout:License : GPLv3+ 2024-06-01T01:42:54.533 INFO:teuthology.orchestra.run.smithi116.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-01T01:42:54.533 INFO:teuthology.orchestra.run.smithi116.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-01T01:42:54.533 INFO:teuthology.orchestra.run.smithi116.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-01T01:42:54.533 INFO:teuthology.orchestra.run.smithi116.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-01T01:42:54.533 INFO:teuthology.orchestra.run.smithi116.stdout:Packager : builder@centos.org 2024-06-01T01:42:54.533 INFO:teuthology.orchestra.run.smithi116.stdout:Vendor : CentOS 2024-06-01T01:42:54.533 INFO:teuthology.orchestra.run.smithi116.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-01T01:42:54.533 INFO:teuthology.orchestra.run.smithi116.stdout:Summary : Support tools for GRUB. 2024-06-01T01:42:54.533 INFO:teuthology.orchestra.run.smithi116.stdout:Description : 2024-06-01T01:42:54.534 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-06-01T01:42:54.534 INFO:teuthology.orchestra.run.smithi116.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-01T01:42:54.534 INFO:teuthology.orchestra.run.smithi116.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-01T01:42:54.534 INFO:teuthology.orchestra.run.smithi116.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-01T01:42:54.534 INFO:teuthology.orchestra.run.smithi116.stdout:hardware devices. 2024-06-01T01:42:54.534 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-06-01T01:42:54.534 INFO:teuthology.orchestra.run.smithi116.stdout:This subpackage provides tools for support of all platforms. 2024-06-01T01:42:54.535 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-01T01:42:54.535 INFO:teuthology.task.kernel:Updating grub on smithi116 to boot 5.14.0-452.el9.x86_64 2024-06-01T01:42:54.535 DEBUG:teuthology.orchestra.run.smithi116:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-01T01:42:54.763 INFO:teuthology.orchestra.run.smithi161.stderr:Generating grub configuration file ... 2024-06-01T01:42:55.149 INFO:teuthology.orchestra.run.smithi116.stderr:Generating grub configuration file ... 2024-06-01T01:42:56.137 INFO:teuthology.orchestra.run.smithi161.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-01T01:42:56.168 INFO:teuthology.orchestra.run.smithi161.stderr:done 2024-06-01T01:42:56.170 DEBUG:teuthology.orchestra.run.smithi161:> mktemp 2024-06-01T01:42:56.198 INFO:teuthology.orchestra.run.smithi161.stdout:/tmp/tmp.C31GMhFXFW 2024-06-01T01:42:56.198 DEBUG:teuthology.orchestra.run.smithi161:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.C31GMhFXFW 2024-06-01T01:42:56.272 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0666 /tmp/tmp.C31GMhFXFW 2024-06-01T01:42:56.408 DEBUG:teuthology.orchestra.remote:smithi161:/tmp/tmp.C31GMhFXFW is 6KB 2024-06-01T01:42:56.429 DEBUG:teuthology.orchestra.run.smithi161:> rm -fr /tmp/tmp.C31GMhFXFW 2024-06-01T01:42:56.443 DEBUG:teuthology.orchestra.run.smithi161:> sudo /bin/ls /boot/loader/entries || true 2024-06-01T01:42:56.513 INFO:teuthology.orchestra.run.smithi161.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-01T01:42:56.513 INFO:teuthology.orchestra.run.smithi161.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-01T01:42:56.514 INFO:teuthology.orchestra.run.smithi161.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-01T01:42:56.514 INFO:teuthology.orchestra.run.smithi161.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64.conf 2024-06-01T01:42:56.515 DEBUG:teuthology.orchestra.run.smithi161:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64 2024-06-01T01:42:56.527 INFO:teuthology.orchestra.run.smithi116.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-01T01:42:56.558 INFO:teuthology.orchestra.run.smithi116.stderr:done 2024-06-01T01:42:56.565 DEBUG:teuthology.orchestra.run.smithi116:> mktemp 2024-06-01T01:42:56.581 INFO:teuthology.orchestra.run.smithi116.stdout:/tmp/tmp.wneQrUy02s 2024-06-01T01:42:56.581 DEBUG:teuthology.orchestra.run.smithi116:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.wneQrUy02s 2024-06-01T01:42:56.655 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 0666 /tmp/tmp.wneQrUy02s 2024-06-01T01:42:56.669 DEBUG:teuthology.orchestra.run.smithi161:> sudo shutdown -r now 2024-06-01T01:42:56.800 DEBUG:teuthology.orchestra.remote:smithi116:/tmp/tmp.wneQrUy02s is 6KB 2024-06-01T01:42:56.850 DEBUG:teuthology.orchestra.run.smithi116:> rm -fr /tmp/tmp.wneQrUy02s 2024-06-01T01:42:56.865 DEBUG:teuthology.orchestra.run.smithi116:> sudo /bin/ls /boot/loader/entries || true 2024-06-01T01:42:56.935 INFO:teuthology.orchestra.run.smithi116.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-01T01:42:56.935 INFO:teuthology.orchestra.run.smithi116.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-01T01:42:56.935 INFO:teuthology.orchestra.run.smithi116.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-01T01:42:56.935 INFO:teuthology.orchestra.run.smithi116.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64.conf 2024-06-01T01:42:56.936 DEBUG:teuthology.orchestra.run.smithi116:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64 2024-06-01T01:42:57.086 DEBUG:teuthology.orchestra.run.smithi116:> sudo shutdown -r now 2024-06-01T01:43:26.695 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-06-01T01:43:26.696 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-06-01T01:43:26.697 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-01T01:43:27.110 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-06-01T01:43:27.110 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-06-01T01:43:27.110 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-01T01:44:26.756 DEBUG:teuthology.orchestra.remote:timed out 2024-06-01T01:44:27.114 DEBUG:teuthology.orchestra.remote:timed out 2024-06-01T01:44:35.766 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-06-01T01:44:35.767 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-01T01:44:36.115 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-06-01T01:44:36.115 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-01T01:44:38.846 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.161 2024-06-01T01:44:50.858 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-06-01T01:44:50.859 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-01T01:44:51.312 DEBUG:teuthology.orchestra.run.smithi161:> true 2024-06-01T01:44:51.823 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-06-01T01:44:51.823 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-452.el9.x86_64"... 2024-06-01T01:44:51.823 DEBUG:teuthology.orchestra.run.smithi161:> uname -r 2024-06-01T01:44:51.877 INFO:teuthology.orchestra.run.smithi161.stdout:5.14.0-452.el9.x86_64 2024-06-01T01:44:51.877 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-452.el9.x86_64 vs 5.14.0-452.el9.x86_64 2024-06-01T01:44:51.877 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-01T01:44:51.878 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-01T01:44:52.878 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-06-01T01:44:52.878 DEBUG:teuthology.orchestra.run.smithi161:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-06-01T01:44:52.953 INFO:teuthology.orchestra.run.smithi161.stdout:ttyS1 2024-06-01T01:44:52.985 DEBUG:teuthology.parallel:result is None 2024-06-01T01:45:07.936 DEBUG:teuthology.orchestra.run.smithi116:> true 2024-06-01T01:45:08.358 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-06-01T01:45:08.358 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-452.el9.x86_64"... 2024-06-01T01:45:08.358 DEBUG:teuthology.orchestra.run.smithi116:> uname -r 2024-06-01T01:45:08.372 INFO:teuthology.orchestra.run.smithi116.stdout:5.14.0-452.el9.x86_64 2024-06-01T01:45:08.373 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-452.el9.x86_64 vs 5.14.0-452.el9.x86_64 2024-06-01T01:45:08.373 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-01T01:45:08.373 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-01T01:45:09.374 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-06-01T01:45:09.374 DEBUG:teuthology.orchestra.run.smithi116:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-06-01T01:45:09.443 INFO:teuthology.orchestra.run.smithi116.stdout:ttyS1 2024-06-01T01:45:09.490 DEBUG:teuthology.parallel:result is None 2024-06-01T01:45:09.490 INFO:teuthology.run_tasks:Running task internal.base... 2024-06-01T01:45:09.498 INFO:teuthology.task.internal:Creating test directory... 2024-06-01T01:45:09.498 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-01T01:45:09.500 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-01T01:45:09.519 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-06-01T01:45:09.525 INFO:teuthology.run_tasks:Running task internal.archive... 2024-06-01T01:45:09.532 INFO:teuthology.task.internal:Creating archive directory... 2024-06-01T01:45:09.532 DEBUG:teuthology.orchestra.run.smithi116:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-01T01:45:09.556 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-01T01:45:09.678 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-06-01T01:45:09.685 INFO:teuthology.task.internal:Enabling coredump saving... 2024-06-01T01:45:09.685 DEBUG:teuthology.orchestra.run.smithi116:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-01T01:45:09.687 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-01T01:45:09.723 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-01T01:45:09.734 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-01T01:45:09.734 INFO:teuthology.orchestra.run.smithi116.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-01T01:45:09.744 INFO:teuthology.orchestra.run.smithi116.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-01T01:45:09.746 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-06-01T01:45:09.753 INFO:teuthology.task.internal:Configuring sudo... 2024-06-01T01:45:09.753 DEBUG:teuthology.orchestra.run.smithi116:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-01T01:45:09.755 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-01T01:45:09.803 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-06-01T01:45:09.813 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-06-01T01:45:09.813 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-01T01:45:09.825 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-01T01:45:09.860 DEBUG:teuthology.orchestra.run.smithi116:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-01T01:45:09.924 DEBUG:teuthology.orchestra.run.smithi116:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-01T01:45:09.967 DEBUG:teuthology.orchestra.run.smithi116:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-01T01:45:10.053 DEBUG:teuthology.orchestra.run.smithi116:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-01T01:45:10.099 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-06-01T01:45:10.099 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-01T01:45:10.167 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-01T01:45:10.213 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-01T01:45:10.262 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-01T01:45:10.347 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-01T01:45:10.390 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-06-01T01:45:10.391 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-01T01:45:10.458 DEBUG:teuthology.orchestra.run.smithi116:> sudo service rsyslog restart 2024-06-01T01:45:10.460 DEBUG:teuthology.orchestra.run.smithi161:> sudo service rsyslog restart 2024-06-01T01:45:10.495 INFO:teuthology.orchestra.run.smithi116.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-01T01:45:10.546 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-01T01:45:10.982 INFO:teuthology.run_tasks:Running task internal.timer... 2024-06-01T01:45:10.989 INFO:teuthology.task.internal:Starting timer... 2024-06-01T01:45:10.989 INFO:teuthology.run_tasks:Running task pcp... 2024-06-01T01:45:10.999 INFO:teuthology.run_tasks:Running task selinux... 2024-06-01T01:45:11.008 DEBUG:teuthology.orchestra.run.smithi116:> sudo service auditd rotate 2024-06-01T01:45:11.090 INFO:teuthology.orchestra.run.smithi116.stdout:Rotating logs: 2024-06-01T01:45:11.092 DEBUG:teuthology.orchestra.run.smithi161:> sudo service auditd rotate 2024-06-01T01:45:11.148 INFO:teuthology.orchestra.run.smithi161.stdout:Rotating logs: 2024-06-01T01:45:11.150 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-06-01T01:45:11.150 DEBUG:teuthology.orchestra.run.smithi116:> /usr/sbin/getenforce 2024-06-01T01:45:11.169 INFO:teuthology.orchestra.run.smithi116.stdout:Permissive 2024-06-01T01:45:11.169 DEBUG:teuthology.orchestra.run.smithi161:> /usr/sbin/getenforce 2024-06-01T01:45:11.214 INFO:teuthology.orchestra.run.smithi161.stdout:Permissive 2024-06-01T01:45:11.214 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi116.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi161.front.sepia.ceph.com': 'permissive'} 2024-06-01T01:45:11.214 DEBUG:teuthology.orchestra.run.smithi116:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-06-01T01:45:11.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-01T01:45:11.239 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-06-01T01:45:11.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-01T01:45:11.268 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-06-01T01:45:11.268 DEBUG:teuthology.orchestra.run.smithi116:> sudo /usr/sbin/setenforce permissive 2024-06-01T01:45:11.309 DEBUG:teuthology.orchestra.run.smithi161:> sudo /usr/sbin/setenforce permissive 2024-06-01T01:45:11.336 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-06-01T01:45:11.346 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-06-01T01:45:11.349 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-06-01T01:45:11.471 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-06-01T01:45:11.483 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-06-01T01:45:11.484 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi116.front.sepia.ceph.com,smithi161.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-06-01T01:49:45.493 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi116.front.sepia.ceph.com'), Remote(name='ubuntu@smithi161.front.sepia.ceph.com')] 2024-06-01T01:49:45.495 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-06-01T01:49:45.496 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-01T01:49:45.579 DEBUG:teuthology.orchestra.run.smithi116:> true 2024-06-01T01:49:45.661 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-06-01T01:49:45.661 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-06-01T01:49:45.662 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-01T01:49:45.753 DEBUG:teuthology.orchestra.run.smithi161:> true 2024-06-01T01:49:45.835 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-06-01T01:49:45.836 INFO:teuthology.run_tasks:Running task clock... 2024-06-01T01:49:45.850 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-06-01T01:49:45.850 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-01T01:49:45.851 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-01T01:49:45.854 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-01T01:49:45.854 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-01T01:49:45.881 INFO:teuthology.orchestra.run.smithi116.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-01T01:49:45.895 INFO:teuthology.orchestra.run.smithi116.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-01T01:49:45.907 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-01T01:49:45.919 INFO:teuthology.orchestra.run.smithi116.stderr:sudo: ntpd: command not found 2024-06-01T01:49:45.923 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-01T01:49:45.930 INFO:teuthology.orchestra.run.smithi116.stdout:506 Cannot talk to daemon 2024-06-01T01:49:45.944 INFO:teuthology.orchestra.run.smithi116.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-01T01:49:45.949 INFO:teuthology.orchestra.run.smithi161.stderr:sudo: ntpd: command not found 2024-06-01T01:49:45.957 INFO:teuthology.orchestra.run.smithi116.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-01T01:49:45.960 INFO:teuthology.orchestra.run.smithi161.stdout:506 Cannot talk to daemon 2024-06-01T01:49:45.975 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-01T01:49:45.989 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-01T01:49:46.018 INFO:teuthology.orchestra.run.smithi116.stderr:bash: line 1: ntpq: command not found 2024-06-01T01:49:46.022 INFO:teuthology.orchestra.run.smithi116.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-01T01:49:46.022 INFO:teuthology.orchestra.run.smithi116.stdout:=============================================================================== 2024-06-01T01:49:46.022 INFO:teuthology.orchestra.run.smithi116.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-01T01:49:46.022 INFO:teuthology.orchestra.run.smithi116.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-01T01:49:46.022 INFO:teuthology.orchestra.run.smithi116.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-01T01:49:46.022 INFO:teuthology.orchestra.run.smithi116.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-01T01:49:46.048 INFO:teuthology.orchestra.run.smithi161.stderr:bash: line 1: ntpq: command not found 2024-06-01T01:49:46.051 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-01T01:49:46.051 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2024-06-01T01:49:46.051 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-01T01:49:46.051 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-01T01:49:46.051 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-01T01:49:46.051 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-01T01:49:46.052 INFO:teuthology.run_tasks:Running task cephadm... 2024-06-01T01:49:46.164 INFO:tasks.cephadm:Config: {'cephadm_branch': 'v17.2.0', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'image': 'quay.io/ceph/ceph:v17.2.0', 'conf': {'global': {'mon election default strategy': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1, 'mgr/cephadm/use_agent': True}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'log-only-match': ['CEPHADM_'], 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'} 2024-06-01T01:49:46.164 INFO:tasks.cephadm:Cluster image is quay.io/ceph/ceph:v17.2.0 2024-06-01T01:49:46.165 INFO:tasks.cephadm:Cluster fsid is 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T01:49:46.165 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-06-01T01:49:46.166 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.116', 'mon.c': '[v2:172.21.15.116:3301,v1:172.21.15.116:6790]', 'mon.b': '172.21.15.161'} 2024-06-01T01:49:46.166 INFO:tasks.cephadm:First mon is mon.a on smithi116 2024-06-01T01:49:46.166 INFO:tasks.cephadm:First mgr is y 2024-06-01T01:49:46.166 INFO:tasks.cephadm:Normalizing hostnames... 2024-06-01T01:49:46.166 DEBUG:teuthology.orchestra.run.smithi116:> sudo hostname $(hostname -s) 2024-06-01T01:49:46.193 DEBUG:teuthology.orchestra.run.smithi161:> sudo hostname $(hostname -s) 2024-06-01T01:49:46.219 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref v17.2.0)... 2024-06-01T01:49:46.219 DEBUG:teuthology.orchestra.run.smithi116:> curl --silent https://raw.githubusercontent.com/ceph/ceph/v17.2.0/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-01T01:49:46.459 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r--. 1 ubuntu ubuntu 320521 Jun 1 01:49 /home/ubuntu/cephtest/cephadm 2024-06-01T01:49:46.459 DEBUG:teuthology.orchestra.run.smithi161:> curl --silent https://raw.githubusercontent.com/ceph/ceph/v17.2.0/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-01T01:49:46.635 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r--. 1 ubuntu ubuntu 320521 Jun 1 01:49 /home/ubuntu/cephtest/cephadm 2024-06-01T01:49:46.636 DEBUG:teuthology.orchestra.run.smithi116:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-01T01:49:46.653 DEBUG:teuthology.orchestra.run.smithi161:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-01T01:49:46.680 INFO:tasks.cephadm:Pulling image quay.io/ceph/ceph:v17.2.0 on all hosts... 2024-06-01T01:49:46.680 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 pull 2024-06-01T01:49:46.695 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 pull 2024-06-01T01:49:47.483 INFO:teuthology.orchestra.run.smithi116.stderr:Pulling container image quay.io/ceph/ceph:v17.2.0... 2024-06-01T01:49:47.583 INFO:teuthology.orchestra.run.smithi161.stderr:Pulling container image quay.io/ceph/ceph:v17.2.0... 2024-06-01T01:50:03.852 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T01:50:03.852 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_version": "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)", 2024-06-01T01:50:03.852 INFO:teuthology.orchestra.run.smithi116.stdout: "image_id": "e1d6a67b021eb077ee22bf650f1a9fb1980a2cf5c36bdb9cba9eac6de8f702d9", 2024-06-01T01:50:03.852 INFO:teuthology.orchestra.run.smithi116.stdout: "repo_digests": [ 2024-06-01T01:50:03.852 INFO:teuthology.orchestra.run.smithi116.stdout: "quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a", 2024-06-01T01:50:03.852 INFO:teuthology.orchestra.run.smithi116.stdout: "quay.io/ceph/ceph@sha256:cb4d698cb769b6aba05bf6ef04f41a7fe694160140347576e13bd9348514b667" 2024-06-01T01:50:03.852 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2024-06-01T01:50:03.853 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T01:50:03.887 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-06-01T01:50:03.887 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_version": "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)", 2024-06-01T01:50:03.888 INFO:teuthology.orchestra.run.smithi161.stdout: "image_id": "e1d6a67b021eb077ee22bf650f1a9fb1980a2cf5c36bdb9cba9eac6de8f702d9", 2024-06-01T01:50:03.888 INFO:teuthology.orchestra.run.smithi161.stdout: "repo_digests": [ 2024-06-01T01:50:03.888 INFO:teuthology.orchestra.run.smithi161.stdout: "quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a", 2024-06-01T01:50:03.888 INFO:teuthology.orchestra.run.smithi161.stdout: "quay.io/ceph/ceph@sha256:cb4d698cb769b6aba05bf6ef04f41a7fe694160140347576e13bd9348514b667" 2024-06-01T01:50:03.888 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-06-01T01:50:03.888 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-06-01T01:50:03.908 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph 2024-06-01T01:50:03.937 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /etc/ceph 2024-06-01T01:50:03.966 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 777 /etc/ceph 2024-06-01T01:50:04.001 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 777 /etc/ceph 2024-06-01T01:50:04.032 INFO:tasks.cephadm:Writing seed config... 2024-06-01T01:50:04.033 INFO:tasks.cephadm: override: [global] mon election default strategy = 3 2024-06-01T01:50:04.033 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-06-01T01:50:04.033 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-06-01T01:50:04.033 INFO:tasks.cephadm: override: [mgr] mgr/cephadm/use_agent = True 2024-06-01T01:50:04.033 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-06-01T01:50:04.033 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-06-01T01:50:04.034 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-06-01T01:50:04.034 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-06-01T01:50:04.034 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-06-01T01:50:04.034 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-06-01T01:50:04.034 DEBUG:teuthology.orchestra.run.smithi116:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-06-01T01:50:04.057 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 393f964a-1fb9-11ef-bc9b-c7b262605968 mon election default strategy = 3 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 mgr/cephadm/use_agent = True [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-06-01T01:50:04.057 DEBUG:teuthology.orchestra.run.smithi116:mon.a> sudo journalctl -f -n 0 -u ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mon.a.service 2024-06-01T01:50:04.099 DEBUG:teuthology.orchestra.run.smithi116:mgr.y> sudo journalctl -f -n 0 -u ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mgr.y.service 2024-06-01T01:50:04.141 INFO:tasks.cephadm:Bootstrapping... 2024-06-01T01:50:04.142 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 -v bootstrap --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.116 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-06-01T01:50:04.334 INFO:teuthology.orchestra.run.smithi116.stderr:-------------------------------------------------------------------------------- 2024-06-01T01:50:04.334 INFO:teuthology.orchestra.run.smithi116.stderr:cephadm ['--image', 'quay.io/ceph/ceph:v17.2.0', '-v', 'bootstrap', '--fsid', '393f964a-1fb9-11ef-bc9b-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.116', '--skip-admin-label'] 2024-06-01T01:50:04.357 INFO:teuthology.orchestra.run.smithi116.stderr:/bin/podman: 5.0.2 2024-06-01T01:50:04.359 INFO:teuthology.orchestra.run.smithi116.stderr:Verifying podman|docker is present... 2024-06-01T01:50:04.380 INFO:teuthology.orchestra.run.smithi116.stderr:/bin/podman: 5.0.2 2024-06-01T01:50:04.382 INFO:teuthology.orchestra.run.smithi116.stderr:Verifying lvm2 is present... 2024-06-01T01:50:04.382 INFO:teuthology.orchestra.run.smithi116.stderr:Verifying time synchronization is in place... 2024-06-01T01:50:04.389 INFO:teuthology.orchestra.run.smithi116.stderr:systemctl: Failed to get unit file state for chrony.service: No such file or directory 2024-06-01T01:50:04.395 INFO:teuthology.orchestra.run.smithi116.stderr:systemctl: inactive 2024-06-01T01:50:04.402 INFO:teuthology.orchestra.run.smithi116.stderr:systemctl: enabled 2024-06-01T01:50:04.408 INFO:teuthology.orchestra.run.smithi116.stderr:systemctl: active 2024-06-01T01:50:04.409 INFO:teuthology.orchestra.run.smithi116.stderr:Unit chronyd.service is enabled and running 2024-06-01T01:50:04.409 INFO:teuthology.orchestra.run.smithi116.stderr:Repeating the final host check... 2024-06-01T01:50:04.429 INFO:teuthology.orchestra.run.smithi116.stderr:/bin/podman: 5.0.2 2024-06-01T01:50:04.431 INFO:teuthology.orchestra.run.smithi116.stderr:podman (/bin/podman) version 5.0.2 is present 2024-06-01T01:50:04.431 INFO:teuthology.orchestra.run.smithi116.stderr:systemctl is present 2024-06-01T01:50:04.431 INFO:teuthology.orchestra.run.smithi116.stderr:lvcreate is present 2024-06-01T01:50:04.438 INFO:teuthology.orchestra.run.smithi116.stderr:systemctl: Failed to get unit file state for chrony.service: No such file or directory 2024-06-01T01:50:04.444 INFO:teuthology.orchestra.run.smithi116.stderr:systemctl: inactive 2024-06-01T01:50:04.450 INFO:teuthology.orchestra.run.smithi116.stderr:systemctl: enabled 2024-06-01T01:50:04.456 INFO:teuthology.orchestra.run.smithi116.stderr:systemctl: active 2024-06-01T01:50:04.457 INFO:teuthology.orchestra.run.smithi116.stderr:Unit chronyd.service is enabled and running 2024-06-01T01:50:04.457 INFO:teuthology.orchestra.run.smithi116.stderr:Host looks OK 2024-06-01T01:50:04.458 INFO:teuthology.orchestra.run.smithi116.stderr:Cluster fsid: 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T01:50:04.458 INFO:teuthology.orchestra.run.smithi116.stderr:Acquiring lock 139683095991296 on /run/cephadm/393f964a-1fb9-11ef-bc9b-c7b262605968.lock 2024-06-01T01:50:04.458 INFO:teuthology.orchestra.run.smithi116.stderr:Lock 139683095991296 acquired on /run/cephadm/393f964a-1fb9-11ef-bc9b-c7b262605968.lock 2024-06-01T01:50:04.458 INFO:teuthology.orchestra.run.smithi116.stderr:Verifying IP 172.21.15.116 port 3300 ... 2024-06-01T01:50:04.458 INFO:teuthology.orchestra.run.smithi116.stderr:Verifying IP 172.21.15.116 port 6789 ... 2024-06-01T01:50:04.458 INFO:teuthology.orchestra.run.smithi116.stderr:Base mon IP is 172.21.15.116, final addrv is [v2:172.21.15.116:3300,v1:172.21.15.116:6789] 2024-06-01T01:50:04.461 INFO:teuthology.orchestra.run.smithi116.stderr:/sbin/ip: default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.116 metric 100 2024-06-01T01:50:04.461 INFO:teuthology.orchestra.run.smithi116.stderr:/sbin/ip: 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.116 metric 100 2024-06-01T01:50:04.463 INFO:teuthology.orchestra.run.smithi116.stderr:/sbin/ip: ::1 dev lo proto kernel metric 256 pref medium 2024-06-01T01:50:04.464 INFO:teuthology.orchestra.run.smithi116.stderr:/sbin/ip: fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-06-01T01:50:04.464 INFO:teuthology.orchestra.run.smithi116.stderr:/sbin/ip: default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1779sec hoplimit 64 pref medium 2024-06-01T01:50:04.466 INFO:teuthology.orchestra.run.smithi116.stderr:/sbin/ip: 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-06-01T01:50:04.466 INFO:teuthology.orchestra.run.smithi116.stderr:/sbin/ip: inet6 ::1/128 scope host 2024-06-01T01:50:04.466 INFO:teuthology.orchestra.run.smithi116.stderr:/sbin/ip: valid_lft forever preferred_lft forever 2024-06-01T01:50:04.466 INFO:teuthology.orchestra.run.smithi116.stderr:/sbin/ip: 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-06-01T01:50:04.467 INFO:teuthology.orchestra.run.smithi116.stderr:/sbin/ip: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2024-06-01T01:50:04.467 INFO:teuthology.orchestra.run.smithi116.stderr:/sbin/ip: valid_lft forever preferred_lft forever 2024-06-01T01:50:04.468 INFO:teuthology.orchestra.run.smithi116.stderr:Mon IP `172.21.15.116` is in CIDR network `172.21.0.0/20` 2024-06-01T01:50:04.468 INFO:teuthology.orchestra.run.smithi116.stderr:- internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-06-01T01:50:04.468 INFO:teuthology.orchestra.run.smithi116.stderr:Pulling container image quay.io/ceph/ceph:v17.2.0... 2024-06-01T01:50:04.569 INFO:teuthology.orchestra.run.smithi116.stderr:/bin/podman: Trying to pull quay.io/ceph/ceph:v17.2.0... 2024-06-01T01:50:05.000 INFO:teuthology.orchestra.run.smithi116.stderr:/bin/podman: Getting image source signatures 2024-06-01T01:50:05.000 INFO:teuthology.orchestra.run.smithi116.stderr:/bin/podman: Copying blob sha256:33ca8fff7868c4dc0c11e09bca97c720eb9cfbab7221216754367dd8de70388a 2024-06-01T01:50:05.000 INFO:teuthology.orchestra.run.smithi116.stderr:/bin/podman: Copying blob sha256:599d07cb321ff0a3c82224e1138fc685793fa69b93ed5780415751a5f7e4b8c2 2024-06-01T01:50:05.000 INFO:teuthology.orchestra.run.smithi116.stderr:/bin/podman: Copying blob sha256:c32ab78b488d0c72f64eded765c0cf6b5bf2c75dab66cb62a9d367fa6ec42513 2024-06-01T01:50:05.000 INFO:teuthology.orchestra.run.smithi116.stderr:/bin/podman: Copying blob sha256:89b4a75bc2d8500f15463747507c9623df43886c134463e7f0527e70900e7a7b 2024-06-01T01:50:05.000 INFO:teuthology.orchestra.run.smithi116.stderr:/bin/podman: Copying blob sha256:a70843738bb77e1ab9c1f85969ebdfa55f178e746be081d1cb4f94011f69eb7c 2024-06-01T01:50:05.000 INFO:teuthology.orchestra.run.smithi116.stderr:/bin/podman: Copying config sha256:e1d6a67b021eb077ee22bf650f1a9fb1980a2cf5c36bdb9cba9eac6de8f702d9 2024-06-01T01:50:05.164 INFO:teuthology.orchestra.run.smithi116.stderr:/bin/podman: Writing manifest to image destination 2024-06-01T01:50:05.326 INFO:teuthology.orchestra.run.smithi116.stderr:/bin/podman: e1d6a67b021eb077ee22bf650f1a9fb1980a2cf5c36bdb9cba9eac6de8f702d9 2024-06-01T01:50:05.671 INFO:teuthology.orchestra.run.smithi116.stderr:ceph: ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable) 2024-06-01T01:50:06.225 INFO:teuthology.orchestra.run.smithi116.stderr:Ceph version: ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable) 2024-06-01T01:50:06.225 INFO:teuthology.orchestra.run.smithi116.stderr:Extracting ceph user uid/gid from container image... 2024-06-01T01:50:06.658 INFO:teuthology.orchestra.run.smithi116.stderr:stat: 167 167 2024-06-01T01:50:07.088 INFO:teuthology.orchestra.run.smithi116.stderr:Creating initial keys... 2024-06-01T01:50:07.480 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph-authtool: AQBPflpm7NuXFxAAqyNwxs+TuKPproCDTHrtMg== 2024-06-01T01:50:12.011 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph-authtool: AQBTflpmEnrnCRAAkw0jCRIy6AqfWAAhfHoaYQ== 2024-06-01T01:50:19.430 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph-authtool: AQBbflpmJvbDFBAAlsNNaBbN6PEYO18tFBCTvw== 2024-06-01T01:50:19.945 INFO:teuthology.orchestra.run.smithi116.stderr:Creating initial monmap... 2024-06-01T01:50:20.353 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-01T01:50:20.353 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/monmaptool: setting min_mon_release = octopus 2024-06-01T01:50:20.353 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: set fsid to 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T01:50:20.353 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-01T01:50:20.902 INFO:teuthology.orchestra.run.smithi116.stderr:monmaptool for a [v2:172.21.15.116:3300,v1:172.21.15.116:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-01T01:50:20.903 INFO:teuthology.orchestra.run.smithi116.stderr:setting min_mon_release = octopus 2024-06-01T01:50:20.903 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/monmaptool: set fsid to 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T01:50:20.903 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-01T01:50:20.903 INFO:teuthology.orchestra.run.smithi116.stderr: 2024-06-01T01:50:20.903 INFO:teuthology.orchestra.run.smithi116.stderr:Creating mon... 2024-06-01T01:50:21.832 INFO:teuthology.orchestra.run.smithi116.stderr:create mon.a on 2024-06-01T01:50:22.065 INFO:teuthology.orchestra.run.smithi116.stderr:systemctl: Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-06-01T01:50:22.210 INFO:teuthology.orchestra.run.smithi116.stderr:systemctl: Created symlink /etc/systemd/system/multi-user.target.wants/ceph-393f964a-1fb9-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-393f964a-1fb9-11ef-bc9b-c7b262605968.target. 2024-06-01T01:50:22.211 INFO:teuthology.orchestra.run.smithi116.stderr:systemctl: Created symlink /etc/systemd/system/ceph.target.wants/ceph-393f964a-1fb9-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-393f964a-1fb9-11ef-bc9b-c7b262605968.target. 2024-06-01T01:50:22.503 INFO:teuthology.orchestra.run.smithi116.stderr:systemctl: Failed to reset failed state of unit ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mon.a.service: Unit ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mon.a.service not loaded. 2024-06-01T01:50:22.511 INFO:teuthology.orchestra.run.smithi116.stderr:systemctl: Created symlink /etc/systemd/system/ceph-393f964a-1fb9-11ef-bc9b-c7b262605968.target.wants/ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mon.a.service → /etc/systemd/system/ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@.service. 2024-06-01T01:50:23.247 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:22 smithi116 podman[24720]: 2024-06-01 01:50:22.811768305 +0000 UTC m=+0.017472908 image pull e1d6a67b021eb077ee22bf650f1a9fb1980a2cf5c36bdb9cba9eac6de8f702d9 quay.io/ceph/ceph:v17.2.0 2024-06-01T01:50:23.248 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:23 smithi116 podman[24720]: 2024-06-01 01:50:23.248543741 +0000 UTC m=+0.454248335 container create a968eb3e44d703468c9bb019d9e87502ca892d743aa770908de342619c69ccd1 (image=quay.io/ceph/ceph:v17.2.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, architecture=x86_64, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, io.openshift.expose-services=, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, version=8, RELEASE=HEAD, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, vendor=Red Hat, Inc., vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, GIT_CLEAN=True, vcs-type=git, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, ceph=True, distribution-scope=public, io.k8s.display-name=CentOS Stream 8, release=754, name=centos-stream, GIT_REPO=https://github.com/ceph/ceph-container.git, io.openshift.tags=base centos centos-stream, CEPH_POINT_RELEASE=-17.2.0, io.buildah.version=1.19.8, build-date=2022-05-03T08:36:31.336870, GIT_BRANCH=HEAD, com.redhat.component=centos-stream-container, maintainer=Guillaume Abrioux ) 2024-06-01T01:50:23.499 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:23 smithi116 podman[24720]: 2024-06-01 01:50:23.450675802 +0000 UTC m=+0.656380404 container init a968eb3e44d703468c9bb019d9e87502ca892d743aa770908de342619c69ccd1 (image=quay.io/ceph/ceph:v17.2.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a, build-date=2022-05-03T08:36:31.336870, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.19.8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.component=centos-stream-container, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, GIT_CLEAN=True, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, name=centos-stream, ceph=True, vendor=Red Hat, Inc., CEPH_POINT_RELEASE=-17.2.0, architecture=x86_64, version=8, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, io.openshift.expose-services=, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, io.k8s.display-name=CentOS Stream 8, io.openshift.tags=base centos centos-stream, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=HEAD, distribution-scope=public, vcs-type=git, release=754) 2024-06-01T01:50:23.499 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:23 smithi116 podman[24720]: 2024-06-01 01:50:23.453884844 +0000 UTC m=+0.659589449 container start a968eb3e44d703468c9bb019d9e87502ca892d743aa770908de342619c69ccd1 (image=quay.io/ceph/ceph:v17.2.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a, GIT_CLEAN=True, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, io.buildah.version=1.19.8, io.openshift.tags=base centos centos-stream, release=754, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, maintainer=Guillaume Abrioux , GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, com.redhat.component=centos-stream-container, name=centos-stream, build-date=2022-05-03T08:36:31.336870, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, vendor=Red Hat, Inc., GIT_BRANCH=HEAD, vcs-type=git, architecture=x86_64, io.k8s.display-name=CentOS Stream 8, io.openshift.expose-services=, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, distribution-scope=public, ceph=True, CEPH_POINT_RELEASE=-17.2.0, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., version=8) 2024-06-01T01:50:23.499 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:23 smithi116 ceph-mon[24739]: mkfs 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T01:50:23.499 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:23 smithi116 ceph-mon[24739]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-01T01:50:23.562 INFO:teuthology.orchestra.run.smithi116.stderr:systemctl: disabled 2024-06-01T01:50:23.569 INFO:teuthology.orchestra.run.smithi116.stderr:systemctl: inactive 2024-06-01T01:50:23.570 INFO:teuthology.orchestra.run.smithi116.stderr:firewalld.service is not enabled 2024-06-01T01:50:23.570 INFO:teuthology.orchestra.run.smithi116.stderr:Not possible to enable service . firewalld.service is not available 2024-06-01T01:50:23.570 INFO:teuthology.orchestra.run.smithi116.stderr:Waiting for mon to start... 2024-06-01T01:50:23.571 INFO:teuthology.orchestra.run.smithi116.stderr:Waiting for mon... 2024-06-01T01:50:23.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:23 smithi116 bash[24720]: a968eb3e44d703468c9bb019d9e87502ca892d743aa770908de342619c69ccd1 2024-06-01T01:50:23.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:23 smithi116 systemd[1]: Started Ceph mon.a for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T01:50:24.108 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: cluster: 2024-06-01T01:50:24.108 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: id: 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T01:50:24.108 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: health: HEALTH_OK 2024-06-01T01:50:24.108 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: 2024-06-01T01:50:24.108 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: services: 2024-06-01T01:50:24.108 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: mon: 1 daemons, quorum a (age 0.615174s) 2024-06-01T01:50:24.108 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: mgr: no daemons active 2024-06-01T01:50:24.108 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: osd: 0 osds: 0 up, 0 in 2024-06-01T01:50:24.108 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: 2024-06-01T01:50:24.108 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: data: 2024-06-01T01:50:24.109 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: pools: 0 pools, 0 pgs 2024-06-01T01:50:24.109 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: objects: 0 objects, 0 B 2024-06-01T01:50:24.109 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: usage: 0 B used, 0 B / 0 B avail 2024-06-01T01:50:24.109 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: pgs: 2024-06-01T01:50:24.109 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: 2024-06-01T01:50:24.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:24 smithi116 ceph-mon[24739]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-01T01:50:24.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:24 smithi116 ceph-mon[24739]: monmap e1: 1 mons at {a=[v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0]} 2024-06-01T01:50:24.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:24 smithi116 ceph-mon[24739]: fsmap 2024-06-01T01:50:24.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:24 smithi116 ceph-mon[24739]: osdmap e1: 0 total, 0 up, 0 in 2024-06-01T01:50:24.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:24 smithi116 ceph-mon[24739]: mgrmap e1: no daemons active 2024-06-01T01:50:24.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:24 smithi116 ceph-mon[24739]: from='client.? 172.21.15.116:0/559507324' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-06-01T01:50:25.515 INFO:teuthology.orchestra.run.smithi116.stderr:mon is available 2024-06-01T01:50:25.515 INFO:teuthology.orchestra.run.smithi116.stderr:Assimilating anything we can from ceph.conf... 2024-06-01T01:50:25.997 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: 2024-06-01T01:50:25.997 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: [global] 2024-06-01T01:50:25.997 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: fsid = 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T01:50:25.997 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: mon_host = [v2:172.21.15.116:3300,v1:172.21.15.116:6789] 2024-06-01T01:50:25.997 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: mon_osd_allow_pg_remap = true 2024-06-01T01:50:25.997 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: mon_osd_allow_primary_affinity = true 2024-06-01T01:50:25.997 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: mon_warn_on_no_sortbitwise = false 2024-06-01T01:50:25.997 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: osd_crush_chooseleaf_type = 0 2024-06-01T01:50:25.997 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: 2024-06-01T01:50:25.998 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: [mgr] 2024-06-01T01:50:25.998 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: mgr/cephadm/use_agent = True 2024-06-01T01:50:25.998 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: mgr/telemetry/nag = false 2024-06-01T01:50:25.998 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: 2024-06-01T01:50:25.998 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: [osd] 2024-06-01T01:50:25.998 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: osd_map_max_advance = 10 2024-06-01T01:50:25.998 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: osd_sloppy_crc = true 2024-06-01T01:50:26.229 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:26 smithi116 ceph-mon[24739]: from='client.? 172.21.15.116:0/1707262858' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-01T01:50:26.230 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:26 smithi116 ceph-mon[24739]: from='client.? 172.21.15.116:0/1707262858' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-06-01T01:50:26.571 INFO:teuthology.orchestra.run.smithi116.stderr:Generating new minimal ceph.conf... 2024-06-01T01:50:27.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:27 smithi116 ceph-mon[24739]: from='client.? 172.21.15.116:0/3494780769' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:50:27.527 INFO:teuthology.orchestra.run.smithi116.stderr:Restarting the monitor... 2024-06-01T01:50:27.574 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:27 smithi116 systemd[1]: Stopping Ceph mon.a for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T01:50:28.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:27 smithi116 bash[24958]: Error: no container with name or ID "ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon.a" found: no such container 2024-06-01T01:50:28.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:27 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a[24735]: 2024-06-01T01:50:27.686+0000 7f5f4399a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-01T01:50:28.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:27 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a[24735]: 2024-06-01T01:50:27.686+0000 7f5f4399a700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-06-01T01:50:28.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:27 smithi116 podman[24970]: 2024-06-01 01:50:27.79000022 +0000 UTC m=+0.205622409 container died a968eb3e44d703468c9bb019d9e87502ca892d743aa770908de342619c69ccd1 (image=quay.io/ceph/ceph:v17.2.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a, vcs-type=git, name=centos-stream, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, release=754, io.openshift.expose-services=, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-17.2.0, GIT_REPO=https://github.com/ceph/ceph-container.git, vendor=Red Hat, Inc., io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, io.openshift.tags=base centos centos-stream, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, version=8, maintainer=Guillaume Abrioux , build-date=2022-05-03T08:36:31.336870, distribution-scope=public, com.redhat.component=centos-stream-container, GIT_CLEAN=True, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, io.k8s.display-name=CentOS Stream 8, ceph=True, io.buildah.version=1.19.8, RELEASE=HEAD, architecture=x86_64, com.redhat.license_terms=https://centos.org/legal/licensing-policy/) 2024-06-01T01:50:28.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 podman[24970]: 2024-06-01 01:50:28.145598951 +0000 UTC m=+0.561221140 container cleanup a968eb3e44d703468c9bb019d9e87502ca892d743aa770908de342619c69ccd1 (image=quay.io/ceph/ceph:v17.2.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a, com.redhat.component=centos-stream-container, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, name=centos-stream, io.openshift.tags=base centos centos-stream, RELEASE=HEAD, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, distribution-scope=public, GIT_REPO=https://github.com/ceph/ceph-container.git, version=8, ceph=True, architecture=x86_64, GIT_BRANCH=HEAD, io.k8s.display-name=CentOS Stream 8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.expose-services=, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, io.buildah.version=1.19.8, vcs-type=git, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, CEPH_POINT_RELEASE=-17.2.0, vendor=Red Hat, Inc., maintainer=Guillaume Abrioux , release=754, GIT_CLEAN=True, build-date=2022-05-03T08:36:31.336870) 2024-06-01T01:50:28.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 bash[24970]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a 2024-06-01T01:50:28.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 bash[24997]: Error: no container with name or ID "ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon.a" found: no such container 2024-06-01T01:50:28.738 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 podman[24982]: 2024-06-01 01:50:28.405047592 +0000 UTC m=+0.613702893 container remove a968eb3e44d703468c9bb019d9e87502ca892d743aa770908de342619c69ccd1 (image=quay.io/ceph/ceph:v17.2.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a, version=8, vcs-type=git, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, maintainer=Guillaume Abrioux , build-date=2022-05-03T08:36:31.336870, architecture=x86_64, RELEASE=HEAD, name=centos-stream, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, io.openshift.expose-services=, io.openshift.tags=base centos centos-stream, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, com.redhat.component=centos-stream-container, io.buildah.version=1.19.8, distribution-scope=public, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., release=754, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, ceph=True, GIT_BRANCH=HEAD, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, GIT_REPO=https://github.com/ceph/ceph-container.git, io.k8s.display-name=CentOS Stream 8, vendor=Red Hat, Inc., CEPH_POINT_RELEASE=-17.2.0, GIT_CLEAN=True) 2024-06-01T01:50:28.738 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mon.a.service: Deactivated successfully. 2024-06-01T01:50:28.738 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 systemd[1]: Stopped Ceph mon.a for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T01:50:28.738 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 systemd[1]: Starting Ceph mon.a for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T01:50:28.738 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 podman[25063]: 2024-06-01 01:50:28.540486114 +0000 UTC m=+0.018686280 image pull e1d6a67b021eb077ee22bf650f1a9fb1980a2cf5c36bdb9cba9eac6de8f702d9 quay.io/ceph/ceph:v17.2.0 2024-06-01T01:50:28.738 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 podman[25063]: 2024-06-01 01:50:28.739407521 +0000 UTC m=+0.217607679 container create c821092a538a3a0bb92488e8e7b3648dcbc35769de1be25daffb5cbc9781b79a (image=quay.io/ceph/ceph:v17.2.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.component=centos-stream-container, GIT_REPO=https://github.com/ceph/ceph-container.git, architecture=x86_64, version=8, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, GIT_BRANCH=HEAD, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, maintainer=Guillaume Abrioux , vendor=Red Hat, Inc., url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, vcs-type=git, io.buildah.version=1.19.8, io.openshift.expose-services=, io.openshift.tags=base centos centos-stream, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, distribution-scope=public, ceph=True, name=centos-stream, CEPH_POINT_RELEASE=-17.2.0, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, release=754, RELEASE=HEAD, GIT_CLEAN=True, io.k8s.display-name=CentOS Stream 8, build-date=2022-05-03T08:36:31.336870) 2024-06-01T01:50:29.023 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 podman[25063]: 2024-06-01 01:50:28.950091723 +0000 UTC m=+0.428291900 container init c821092a538a3a0bb92488e8e7b3648dcbc35769de1be25daffb5cbc9781b79a (image=quay.io/ceph/ceph:v17.2.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, name=centos-stream, maintainer=Guillaume Abrioux , vendor=Red Hat, Inc., version=8, RELEASE=HEAD, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.component=centos-stream-container, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, ceph=True, distribution-scope=public, io.k8s.display-name=CentOS Stream 8, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, release=754, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, GIT_REPO=https://github.com/ceph/ceph-container.git, io.openshift.expose-services=, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, vcs-type=git, architecture=x86_64, io.openshift.tags=base centos centos-stream, io.buildah.version=1.19.8, GIT_CLEAN=True, GIT_BRANCH=HEAD, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, CEPH_POINT_RELEASE=-17.2.0, build-date=2022-05-03T08:36:31.336870) 2024-06-01T01:50:29.023 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 podman[25063]: 2024-06-01 01:50:28.953272108 +0000 UTC m=+0.431472274 container start c821092a538a3a0bb92488e8e7b3648dcbc35769de1be25daffb5cbc9781b79a (image=quay.io/ceph/ceph:v17.2.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a, vcs-type=git, GIT_REPO=https://github.com/ceph/ceph-container.git, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vendor=Red Hat, Inc., CEPH_POINT_RELEASE=-17.2.0, version=8, io.buildah.version=1.19.8, io.openshift.expose-services=, io.k8s.display-name=CentOS Stream 8, build-date=2022-05-03T08:36:31.336870, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, name=centos-stream, io.openshift.tags=base centos centos-stream, distribution-scope=public, GIT_BRANCH=HEAD, com.redhat.component=centos-stream-container, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, maintainer=Guillaume Abrioux , ceph=True, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, architecture=x86_64, release=754, RELEASE=HEAD, GIT_CLEAN=True) 2024-06-01T01:50:29.023 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: set uid:gid to 167:167 (ceph:ceph) 2024-06-01T01:50:29.023 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable), process ceph-mon, pid 2 2024-06-01T01:50:29.023 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: pidfile_write: ignore empty --pid-file 2024-06-01T01:50:29.024 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: load: jerasure load: lrc 2024-06-01T01:50:29.024 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: RocksDB version: 6.15.5 2024-06-01T01:50:29.024 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-06-01T01:50:29.024 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Compile date Apr 18 2022 2024-06-01T01:50:29.024 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: DB SUMMARY 2024-06-01T01:50:29.024 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: DB Session ID: 8SYXP0UBJ7NHTFXKPU2R 2024-06-01T01:50:29.024 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: CURRENT file: CURRENT 2024-06-01T01:50:29.024 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: IDENTITY file: IDENTITY 2024-06-01T01:50:29.025 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-06-01T01:50:29.025 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-06-01T01:50:29.025 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000010.log size: 82661 ; 2024-06-01T01:50:29.025 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.error_if_exists: 0 2024-06-01T01:50:29.025 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.create_if_missing: 0 2024-06-01T01:50:29.025 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.paranoid_checks: 1 2024-06-01T01:50:29.025 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-01T01:50:29.025 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.env: 0x55643d8c6860 2024-06-01T01:50:29.025 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.fs: Posix File System 2024-06-01T01:50:29.025 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.info_log: 0x55643f7b7dc0 2024-06-01T01:50:29.025 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.max_file_opening_threads: 16 2024-06-01T01:50:29.025 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.statistics: (nil) 2024-06-01T01:50:29.025 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.use_fsync: 0 2024-06-01T01:50:29.025 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.max_log_file_size: 0 2024-06-01T01:50:29.026 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-01T01:50:29.026 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-01T01:50:29.026 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.keep_log_file_num: 1000 2024-06-01T01:50:29.026 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.recycle_log_file_num: 0 2024-06-01T01:50:29.026 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.allow_fallocate: 1 2024-06-01T01:50:29.026 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.allow_mmap_reads: 0 2024-06-01T01:50:29.026 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.allow_mmap_writes: 0 2024-06-01T01:50:29.026 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.use_direct_reads: 0 2024-06-01T01:50:29.026 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-01T01:50:29.026 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.create_missing_column_families: 0 2024-06-01T01:50:29.026 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.db_log_dir: 2024-06-01T01:50:29.026 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-06-01T01:50:29.027 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-01T01:50:29.027 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-01T01:50:29.027 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-01T01:50:29.027 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-01T01:50:29.027 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-01T01:50:29.027 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-01T01:50:29.027 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.advise_random_on_open: 1 2024-06-01T01:50:29.028 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.db_write_buffer_size: 0 2024-06-01T01:50:29.028 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.write_buffer_manager: 0x55643f8a8240 2024-06-01T01:50:29.028 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-01T01:50:29.028 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-06-01T01:50:29.028 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-01T01:50:29.028 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-01T01:50:29.028 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.rate_limiter: (nil) 2024-06-01T01:50:29.028 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-01T01:50:29.028 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.wal_recovery_mode: 2 2024-06-01T01:50:29.028 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.enable_thread_tracking: 0 2024-06-01T01:50:29.029 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.enable_pipelined_write: 0 2024-06-01T01:50:29.029 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.unordered_write: 0 2024-06-01T01:50:29.029 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-01T01:50:29.029 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-01T01:50:29.029 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-01T01:50:29.029 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-01T01:50:29.029 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.row_cache: None 2024-06-01T01:50:29.029 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.wal_filter: None 2024-06-01T01:50:29.029 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-01T01:50:29.029 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.allow_ingest_behind: 0 2024-06-01T01:50:29.029 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.preserve_deletes: 0 2024-06-01T01:50:29.029 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.two_write_queues: 0 2024-06-01T01:50:29.029 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.manual_wal_flush: 0 2024-06-01T01:50:29.029 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.atomic_flush: 0 2024-06-01T01:50:29.029 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-01T01:50:29.030 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-01T01:50:29.030 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-01T01:50:29.030 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.log_readahead_size: 0 2024-06-01T01:50:29.030 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-01T01:50:29.030 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.best_efforts_recovery: 0 2024-06-01T01:50:29.030 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-01T01:50:29.030 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-01T01:50:29.030 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.allow_data_in_errors: 0 2024-06-01T01:50:29.030 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.db_host_id: __hostname__ 2024-06-01T01:50:29.030 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.max_background_jobs: 2 2024-06-01T01:50:29.030 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.max_background_compactions: -1 2024-06-01T01:50:29.030 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.max_subcompactions: 1 2024-06-01T01:50:29.030 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-01T01:50:29.030 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-01T01:50:29.031 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-01T01:50:29.031 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.max_total_wal_size: 0 2024-06-01T01:50:29.031 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-01T01:50:29.031 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-01T01:50:29.031 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-01T01:50:29.031 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-01T01:50:29.031 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.max_open_files: -1 2024-06-01T01:50:29.031 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.bytes_per_sync: 0 2024-06-01T01:50:29.031 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-01T01:50:29.031 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-01T01:50:29.031 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.compaction_readahead_size: 0 2024-06-01T01:50:29.031 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.max_background_flushes: -1 2024-06-01T01:50:29.031 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Compression algorithms supported: 2024-06-01T01:50:29.031 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-01T01:50:29.032 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: kZSTD supported: 0 2024-06-01T01:50:29.032 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: kXpressCompression supported: 0 2024-06-01T01:50:29.032 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: kLZ4HCCompression supported: 1 2024-06-01T01:50:29.032 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: kLZ4Compression supported: 1 2024-06-01T01:50:29.032 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: kBZip2Compression supported: 0 2024-06-01T01:50:29.032 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: kZlibCompression supported: 1 2024-06-01T01:50:29.032 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: kSnappyCompression supported: 1 2024-06-01T01:50:29.032 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-01T01:50:29.032 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: [db/version_set.cc:4725] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 2024-06-01T01:50:29.032 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: [db/column_family.cc:597] --------------- Options for column family [default]: 2024-06-01T01:50:29.032 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-01T01:50:29.032 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.merge_operator: 2024-06-01T01:50:29.032 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.compaction_filter: None 2024-06-01T01:50:29.032 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.compaction_filter_factory: None 2024-06-01T01:50:29.033 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.sst_partitioner_factory: None 2024-06-01T01:50:29.033 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-01T01:50:29.033 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-01T01:50:29.033 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55643f783d00) 2024-06-01T01:50:29.033 INFO:journalctl@ceph.mon.a.smithi116.stdout: cache_index_and_filter_blocks: 1 2024-06-01T01:50:29.033 INFO:journalctl@ceph.mon.a.smithi116.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-01T01:50:29.033 INFO:journalctl@ceph.mon.a.smithi116.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-01T01:50:29.033 INFO:journalctl@ceph.mon.a.smithi116.stdout: pin_top_level_index_and_filter: 1 2024-06-01T01:50:29.033 INFO:journalctl@ceph.mon.a.smithi116.stdout: index_type: 0 2024-06-01T01:50:29.033 INFO:journalctl@ceph.mon.a.smithi116.stdout: data_block_index_type: 0 2024-06-01T01:50:29.033 INFO:journalctl@ceph.mon.a.smithi116.stdout: index_shortening: 1 2024-06-01T01:50:29.033 INFO:journalctl@ceph.mon.a.smithi116.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-01T01:50:29.033 INFO:journalctl@ceph.mon.a.smithi116.stdout: hash_index_allow_collision: 1 2024-06-01T01:50:29.033 INFO:journalctl@ceph.mon.a.smithi116.stdout: checksum: 1 2024-06-01T01:50:29.034 INFO:journalctl@ceph.mon.a.smithi116.stdout: no_block_cache: 0 2024-06-01T01:50:29.034 INFO:journalctl@ceph.mon.a.smithi116.stdout: block_cache: 0x55643f7ee170 2024-06-01T01:50:29.034 INFO:journalctl@ceph.mon.a.smithi116.stdout: block_cache_name: BinnedLRUCache 2024-06-01T01:50:29.034 INFO:journalctl@ceph.mon.a.smithi116.stdout: block_cache_options: 2024-06-01T01:50:29.034 INFO:journalctl@ceph.mon.a.smithi116.stdout: capacity : 536870912 2024-06-01T01:50:29.034 INFO:journalctl@ceph.mon.a.smithi116.stdout: num_shard_bits : 4 2024-06-01T01:50:29.034 INFO:journalctl@ceph.mon.a.smithi116.stdout: strict_capacity_limit : 0 2024-06-01T01:50:29.034 INFO:journalctl@ceph.mon.a.smithi116.stdout: high_pri_pool_ratio: 0.000 2024-06-01T01:50:29.034 INFO:journalctl@ceph.mon.a.smithi116.stdout: block_cache_compressed: (nil) 2024-06-01T01:50:29.034 INFO:journalctl@ceph.mon.a.smithi116.stdout: persistent_cache: (nil) 2024-06-01T01:50:29.034 INFO:journalctl@ceph.mon.a.smithi116.stdout: block_size: 4096 2024-06-01T01:50:29.034 INFO:journalctl@ceph.mon.a.smithi116.stdout: block_size_deviation: 10 2024-06-01T01:50:29.034 INFO:journalctl@ceph.mon.a.smithi116.stdout: block_restart_interval: 16 2024-06-01T01:50:29.034 INFO:journalctl@ceph.mon.a.smithi116.stdout: index_block_restart_interval: 1 2024-06-01T01:50:29.035 INFO:journalctl@ceph.mon.a.smithi116.stdout: metadata_block_size: 4096 2024-06-01T01:50:29.035 INFO:journalctl@ceph.mon.a.smithi116.stdout: partition_filters: 0 2024-06-01T01:50:29.035 INFO:journalctl@ceph.mon.a.smithi116.stdout: use_delta_encoding: 1 2024-06-01T01:50:29.035 INFO:journalctl@ceph.mon.a.smithi116.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-06-01T01:50:29.035 INFO:journalctl@ceph.mon.a.smithi116.stdout: whole_key_filtering: 1 2024-06-01T01:50:29.035 INFO:journalctl@ceph.mon.a.smithi116.stdout: verify_compression: 0 2024-06-01T01:50:29.035 INFO:journalctl@ceph.mon.a.smithi116.stdout: read_amp_bytes_per_bit: 0 2024-06-01T01:50:29.035 INFO:journalctl@ceph.mon.a.smithi116.stdout: format_version: 4 2024-06-01T01:50:29.035 INFO:journalctl@ceph.mon.a.smithi116.stdout: enable_index_compression: 1 2024-06-01T01:50:29.035 INFO:journalctl@ceph.mon.a.smithi116.stdout: block_align: 0 2024-06-01T01:50:29.035 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.write_buffer_size: 33554432 2024-06-01T01:50:29.035 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.max_write_buffer_number: 2 2024-06-01T01:50:29.035 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.compression: NoCompression 2024-06-01T01:50:29.035 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.bottommost_compression: Disabled 2024-06-01T01:50:29.036 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.prefix_extractor: nullptr 2024-06-01T01:50:29.036 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-01T01:50:29.036 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.num_levels: 7 2024-06-01T01:50:29.036 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-01T01:50:29.036 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-01T01:50:29.036 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-01T01:50:29.036 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-01T01:50:29.036 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-01T01:50:29.036 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-01T01:50:29.036 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-01T01:50:29.036 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-01T01:50:29.036 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-01T01:50:29.036 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-01T01:50:29.036 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-01T01:50:29.037 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.compression_opts.level: 32767 2024-06-01T01:50:29.037 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.compression_opts.strategy: 0 2024-06-01T01:50:29.037 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-01T01:50:29.037 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-01T01:50:29.037 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-01T01:50:29.037 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.compression_opts.enabled: false 2024-06-01T01:50:29.037 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-01T01:50:29.037 INFO:teuthology.orchestra.run.smithi116.stderr:Setting mon public_network to 172.21.0.0/20 2024-06-01T01:50:29.038 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-01T01:50:29.038 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-01T01:50:29.038 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.target_file_size_base: 67108864 2024-06-01T01:50:29.038 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-01T01:50:29.038 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-01T01:50:29.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-01T01:50:29.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-01T01:50:29.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-01T01:50:29.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-01T01:50:29.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-01T01:50:29.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-01T01:50:29.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-01T01:50:29.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-01T01:50:29.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-01T01:50:29.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-01T01:50:29.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-01T01:50:29.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.arena_block_size: 4194304 2024-06-01T01:50:29.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-01T01:50:29.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-01T01:50:29.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-06-01T01:50:29.040 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.disable_auto_compactions: 0 2024-06-01T01:50:29.040 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-01T01:50:29.040 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-01T01:50:29.040 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-01T01:50:29.040 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-01T01:50:29.040 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-01T01:50:29.040 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-01T01:50:29.040 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-01T01:50:29.040 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-01T01:50:29.040 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-01T01:50:29.040 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-01T01:50:29.040 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.table_properties_collectors: 2024-06-01T01:50:29.040 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.inplace_update_support: 0 2024-06-01T01:50:29.041 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-01T01:50:29.041 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-01T01:50:29.041 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-01T01:50:29.041 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-01T01:50:29.041 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.bloom_locality: 0 2024-06-01T01:50:29.041 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.max_successive_merges: 0 2024-06-01T01:50:29.041 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-01T01:50:29.041 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.paranoid_file_checks: 0 2024-06-01T01:50:29.041 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.force_consistency_checks: 1 2024-06-01T01:50:29.041 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.report_bg_io_stats: 0 2024-06-01T01:50:29.041 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.ttl: 2592000 2024-06-01T01:50:29.041 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-01T01:50:29.041 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.enable_blob_files: false 2024-06-01T01:50:29.041 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.min_blob_size: 0 2024-06-01T01:50:29.041 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.blob_file_size: 268435456 2024-06-01T01:50:29.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-01T01:50:29.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-01T01:50:29.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-01T01:50:29.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: [db/version_set.cc:4773] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-06-01T01:50:29.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: [db/version_set.cc:4782] Column family [default] (ID 0), log number is 5 2024-06-01T01:50:29.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: [db/version_set.cc:4083] Creating manifest 13 2024-06-01T01:50:29.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717206628978705, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-06-01T01:50:29.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: [db/db_impl/db_impl_open.cc:847] Recovering log #10 mode 2 2024-06-01T01:50:29.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: [table/block_based/filter_policy.cc:996] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-06-01T01:50:29.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717206628979402, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 79211, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 77474, "index_size": 225, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10027, "raw_average_key_size": 47, "raw_value_size": 71783, "raw_average_value_size": 340, "num_data_blocks": 10, "num_entries": 211, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1717206628, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "5755b057-1280-4380-8230-b004a6acc7a7", "db_session_id": "8SYXP0UBJ7NHTFXKPU2R"}} 2024-06-01T01:50:29.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: [db/version_set.cc:4083] Creating manifest 15 2024-06-01T01:50:29.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717206628979710, "job": 1, "event": "recovery_finished"} 2024-06-01T01:50:29.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: [file/delete_scheduler.cc:73] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-01T01:50:29.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: [db/db_impl/db_impl_open.cc:1701] SstFileManager instance 0x55643f7d4a80 2024-06-01T01:50:29.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: DB pointer 0x55643f7e4000 2024-06-01T01:50:29.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: [db/db_impl/db_impl.cc:902] ------- DUMPING STATS ------- 2024-06-01T01:50:29.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-06-01T01:50:29.043 INFO:journalctl@ceph.mon.a.smithi116.stdout: ** DB Stats ** 2024-06-01T01:50:29.043 INFO:journalctl@ceph.mon.a.smithi116.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-01T01:50:29.043 INFO:journalctl@ceph.mon.a.smithi116.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-01T01:50:29.043 INFO:journalctl@ceph.mon.a.smithi116.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-01T01:50:29.043 INFO:journalctl@ceph.mon.a.smithi116.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-01T01:50:29.043 INFO:journalctl@ceph.mon.a.smithi116.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-01T01:50:29.043 INFO:journalctl@ceph.mon.a.smithi116.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-06-01T01:50:29.043 INFO:journalctl@ceph.mon.a.smithi116.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-01T01:50:29.044 INFO:journalctl@ceph.mon.a.smithi116.stdout: 2024-06-01T01:50:29.044 INFO:journalctl@ceph.mon.a.smithi116.stdout: ** Compaction Stats [default] ** 2024-06-01T01:50:29.044 INFO:journalctl@ceph.mon.a.smithi116.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-06-01T01:50:29.044 INFO:journalctl@ceph.mon.a.smithi116.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-01T01:50:29.044 INFO:journalctl@ceph.mon.a.smithi116.stdout: L0 2/0 79.11 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 164.6 0.00 0.00 1 0.000 0 0 2024-06-01T01:50:29.044 INFO:journalctl@ceph.mon.a.smithi116.stdout: Sum 2/0 79.11 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 164.6 0.00 0.00 1 0.000 0 0 2024-06-01T01:50:29.044 INFO:journalctl@ceph.mon.a.smithi116.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 164.6 0.00 0.00 1 0.000 0 0 2024-06-01T01:50:29.044 INFO:journalctl@ceph.mon.a.smithi116.stdout: 2024-06-01T01:50:29.044 INFO:journalctl@ceph.mon.a.smithi116.stdout: ** Compaction Stats [default] ** 2024-06-01T01:50:29.044 INFO:journalctl@ceph.mon.a.smithi116.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-06-01T01:50:29.044 INFO:journalctl@ceph.mon.a.smithi116.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-01T01:50:29.044 INFO:journalctl@ceph.mon.a.smithi116.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 164.6 0.00 0.00 1 0.000 0 0 2024-06-01T01:50:29.044 INFO:journalctl@ceph.mon.a.smithi116.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-01T01:50:29.044 INFO:journalctl@ceph.mon.a.smithi116.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-01T01:50:29.045 INFO:journalctl@ceph.mon.a.smithi116.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-01T01:50:29.045 INFO:journalctl@ceph.mon.a.smithi116.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-01T01:50:29.045 INFO:journalctl@ceph.mon.a.smithi116.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-01T01:50:29.045 INFO:journalctl@ceph.mon.a.smithi116.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-01T01:50:29.045 INFO:journalctl@ceph.mon.a.smithi116.stdout: Cumulative compaction: 0.00 GB write, 25.94 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-01T01:50:29.045 INFO:journalctl@ceph.mon.a.smithi116.stdout: Interval compaction: 0.00 GB write, 25.94 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-01T01:50:29.045 INFO:journalctl@ceph.mon.a.smithi116.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-01T01:50:29.045 INFO:journalctl@ceph.mon.a.smithi116.stdout: 2024-06-01T01:50:29.045 INFO:journalctl@ceph.mon.a.smithi116.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-01T01:50:29.045 INFO:journalctl@ceph.mon.a.smithi116.stdout: 2024-06-01T01:50:29.045 INFO:journalctl@ceph.mon.a.smithi116.stdout: ** Compaction Stats [default] ** 2024-06-01T01:50:29.045 INFO:journalctl@ceph.mon.a.smithi116.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-06-01T01:50:29.045 INFO:journalctl@ceph.mon.a.smithi116.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-01T01:50:29.045 INFO:journalctl@ceph.mon.a.smithi116.stdout: L0 2/0 79.11 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 164.6 0.00 0.00 1 0.000 0 0 2024-06-01T01:50:29.046 INFO:journalctl@ceph.mon.a.smithi116.stdout: Sum 2/0 79.11 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 164.6 0.00 0.00 1 0.000 0 0 2024-06-01T01:50:29.046 INFO:journalctl@ceph.mon.a.smithi116.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-06-01T01:50:29.046 INFO:journalctl@ceph.mon.a.smithi116.stdout: 2024-06-01T01:50:29.046 INFO:journalctl@ceph.mon.a.smithi116.stdout: ** Compaction Stats [default] ** 2024-06-01T01:50:29.046 INFO:journalctl@ceph.mon.a.smithi116.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-06-01T01:50:29.046 INFO:journalctl@ceph.mon.a.smithi116.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-01T01:50:29.046 INFO:journalctl@ceph.mon.a.smithi116.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 164.6 0.00 0.00 1 0.000 0 0 2024-06-01T01:50:29.046 INFO:journalctl@ceph.mon.a.smithi116.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-01T01:50:29.046 INFO:journalctl@ceph.mon.a.smithi116.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-01T01:50:29.046 INFO:journalctl@ceph.mon.a.smithi116.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-01T01:50:29.046 INFO:journalctl@ceph.mon.a.smithi116.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-01T01:50:29.046 INFO:journalctl@ceph.mon.a.smithi116.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-01T01:50:29.046 INFO:journalctl@ceph.mon.a.smithi116.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-01T01:50:29.046 INFO:journalctl@ceph.mon.a.smithi116.stdout: Cumulative compaction: 0.00 GB write, 25.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-01T01:50:29.047 INFO:journalctl@ceph.mon.a.smithi116.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-01T01:50:29.047 INFO:journalctl@ceph.mon.a.smithi116.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-01T01:50:29.047 INFO:journalctl@ceph.mon.a.smithi116.stdout: 2024-06-01T01:50:29.047 INFO:journalctl@ceph.mon.a.smithi116.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-01T01:50:29.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: starting mon.a rank 0 at public addrs [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] at bind addrs [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T01:50:29.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: mon.a@-1(???) e1 preinit fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T01:50:29.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: mon.a@-1(???).mds e1 new map 2024-06-01T01:50:29.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: mon.a@-1(???).mds e1 print_map 2024-06-01T01:50:29.047 INFO:journalctl@ceph.mon.a.smithi116.stdout: e1 2024-06-01T01:50:29.047 INFO:journalctl@ceph.mon.a.smithi116.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-01T01:50:29.047 INFO:journalctl@ceph.mon.a.smithi116.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-01T01:50:29.047 INFO:journalctl@ceph.mon.a.smithi116.stdout: legacy client fscid: -1 2024-06-01T01:50:29.047 INFO:journalctl@ceph.mon.a.smithi116.stdout: 2024-06-01T01:50:29.047 INFO:journalctl@ceph.mon.a.smithi116.stdout: No filesystems configured 2024-06-01T01:50:29.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-01T01:50:29.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-01T01:50:29.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-01T01:50:29.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-01T01:50:29.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:28 smithi116 ceph-mon[25082]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-06-01T01:50:29.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:29 smithi116 bash[25063]: c821092a538a3a0bb92488e8e7b3648dcbc35769de1be25daffb5cbc9781b79a 2024-06-01T01:50:29.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:29 smithi116 systemd[1]: Started Ceph mon.a for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T01:50:29.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:29 smithi116 ceph-mon[25082]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-01T01:50:29.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:29 smithi116 ceph-mon[25082]: monmap e1: 1 mons at {a=[v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0]} 2024-06-01T01:50:29.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:29 smithi116 ceph-mon[25082]: fsmap 2024-06-01T01:50:29.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:29 smithi116 ceph-mon[25082]: osdmap e1: 0 total, 0 up, 0 in 2024-06-01T01:50:29.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:29 smithi116 ceph-mon[25082]: mgrmap e1: no daemons active 2024-06-01T01:50:30.091 INFO:teuthology.orchestra.run.smithi116.stderr:Wrote config to /etc/ceph/ceph.conf 2024-06-01T01:50:30.091 INFO:teuthology.orchestra.run.smithi116.stderr:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-06-01T01:50:30.091 INFO:teuthology.orchestra.run.smithi116.stderr:Creating mgr... 2024-06-01T01:50:30.092 INFO:teuthology.orchestra.run.smithi116.stderr:Verifying port 9283 ... 2024-06-01T01:50:30.260 INFO:teuthology.orchestra.run.smithi116.stderr:systemctl: Failed to reset failed state of unit ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mgr.y.service: Unit ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mgr.y.service not loaded. 2024-06-01T01:50:30.267 INFO:teuthology.orchestra.run.smithi116.stderr:systemctl: Created symlink /etc/systemd/system/ceph-393f964a-1fb9-11ef-bc9b-c7b262605968.target.wants/ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@.service. 2024-06-01T01:50:30.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:30 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/224777694' entity='client.admin' 2024-06-01T01:50:30.942 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:30 smithi116 podman[25297]: 2024-06-01 01:50:30.845863389 +0000 UTC m=+0.316083450 container start 3c1beb78ef72d2fbfda7f39828beb3f0f8e0a99a40959351b7d023d5bb717fba (image=quay.io/ceph/ceph:v17.2.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y, io.k8s.display-name=CentOS Stream 8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., maintainer=Guillaume Abrioux , version=8, io.buildah.version=1.19.8, name=centos-stream, RELEASE=HEAD, distribution-scope=public, com.redhat.component=centos-stream-container, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, io.openshift.tags=base centos centos-stream, vendor=Red Hat, Inc., architecture=x86_64, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, io.openshift.expose-services=, release=754, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, build-date=2022-05-03T08:36:31.336870, vcs-type=git, CEPH_POINT_RELEASE=-17.2.0, ceph=True) 2024-06-01T01:50:30.943 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:30 smithi116 bash[25297]: 3c1beb78ef72d2fbfda7f39828beb3f0f8e0a99a40959351b7d023d5bb717fba 2024-06-01T01:50:30.955 INFO:teuthology.orchestra.run.smithi116.stderr:systemctl: disabled 2024-06-01T01:50:30.963 INFO:teuthology.orchestra.run.smithi116.stderr:systemctl: inactive 2024-06-01T01:50:30.964 INFO:teuthology.orchestra.run.smithi116.stderr:firewalld.service is not enabled 2024-06-01T01:50:30.964 INFO:teuthology.orchestra.run.smithi116.stderr:Not possible to enable service . firewalld.service is not available 2024-06-01T01:50:30.973 INFO:teuthology.orchestra.run.smithi116.stderr:systemctl: disabled 2024-06-01T01:50:30.979 INFO:teuthology.orchestra.run.smithi116.stderr:systemctl: inactive 2024-06-01T01:50:30.980 INFO:teuthology.orchestra.run.smithi116.stderr:firewalld.service is not enabled 2024-06-01T01:50:30.980 INFO:teuthology.orchestra.run.smithi116.stderr:Not possible to open ports <[9283]>. firewalld.service is not available 2024-06-01T01:50:30.980 INFO:teuthology.orchestra.run.smithi116.stderr:Waiting for mgr to start... 2024-06-01T01:50:30.980 INFO:teuthology.orchestra.run.smithi116.stderr:Waiting for mgr... 2024-06-01T01:50:31.222 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:30 smithi116 systemd[1]: Started Ceph mgr.y for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T01:50:31.222 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:30 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:30.991+0000 7fdef7d41000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-01T01:50:31.222 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:31 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:31.124+0000 7fdef7d41000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-01T01:50:31.222 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:31 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:31.222+0000 7fdef7d41000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-01T01:50:31.505 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: 2024-06-01T01:50:31.506 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: { 2024-06-01T01:50:31.506 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "fsid": "393f964a-1fb9-11ef-bc9b-c7b262605968", 2024-06-01T01:50:31.506 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "health": { 2024-06-01T01:50:31.506 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-06-01T01:50:31.506 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "checks": {}, 2024-06-01T01:50:31.506 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "mutes": [] 2024-06-01T01:50:31.506 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: }, 2024-06-01T01:50:31.506 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-06-01T01:50:31.506 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "quorum": [ 2024-06-01T01:50:31.506 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: 0 2024-06-01T01:50:31.506 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: ], 2024-06-01T01:50:31.507 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "quorum_names": [ 2024-06-01T01:50:31.507 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "a" 2024-06-01T01:50:31.507 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: ], 2024-06-01T01:50:31.507 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "quorum_age": 2, 2024-06-01T01:50:31.507 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "monmap": { 2024-06-01T01:50:31.507 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "epoch": 1, 2024-06-01T01:50:31.507 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "min_mon_release_name": "quincy", 2024-06-01T01:50:31.507 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_mons": 1 2024-06-01T01:50:31.507 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: }, 2024-06-01T01:50:31.507 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "osdmap": { 2024-06-01T01:50:31.507 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "epoch": 1, 2024-06-01T01:50:31.507 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_osds": 0, 2024-06-01T01:50:31.507 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-06-01T01:50:31.507 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-06-01T01:50:31.508 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-06-01T01:50:31.508 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-06-01T01:50:31.511 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-06-01T01:50:31.511 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: }, 2024-06-01T01:50:31.511 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "pgmap": { 2024-06-01T01:50:31.511 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-06-01T01:50:31.511 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-06-01T01:50:31.511 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_pools": 0, 2024-06-01T01:50:31.512 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_objects": 0, 2024-06-01T01:50:31.512 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-06-01T01:50:31.512 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-06-01T01:50:31.512 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-06-01T01:50:31.512 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "bytes_total": 0 2024-06-01T01:50:31.512 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: }, 2024-06-01T01:50:31.512 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "fsmap": { 2024-06-01T01:50:31.512 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "epoch": 1, 2024-06-01T01:50:31.512 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "by_rank": [], 2024-06-01T01:50:31.512 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "up:standby": 0 2024-06-01T01:50:31.512 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: }, 2024-06-01T01:50:31.512 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "mgrmap": { 2024-06-01T01:50:31.512 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "available": false, 2024-06-01T01:50:31.513 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-06-01T01:50:31.513 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "modules": [ 2024-06-01T01:50:31.513 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "iostat", 2024-06-01T01:50:31.513 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "nfs", 2024-06-01T01:50:31.513 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "restful" 2024-06-01T01:50:31.513 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: ], 2024-06-01T01:50:31.513 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "services": {} 2024-06-01T01:50:31.513 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: }, 2024-06-01T01:50:31.513 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "servicemap": { 2024-06-01T01:50:31.513 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "epoch": 1, 2024-06-01T01:50:31.513 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "modified": "2024-06-01T01:50:23.494000+0000", 2024-06-01T01:50:31.513 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "services": {} 2024-06-01T01:50:31.513 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: }, 2024-06-01T01:50:31.513 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "progress_events": {} 2024-06-01T01:50:31.514 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: } 2024-06-01T01:50:31.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:31 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/4184357834' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-01T01:50:31.796 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:31 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:31.545+0000 7fdef7d41000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-01T01:50:31.796 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:31 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:31.627+0000 7fdef7d41000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-01T01:50:32.167 INFO:teuthology.orchestra.run.smithi116.stderr:mgr not available, waiting (1/15)... 2024-06-01T01:50:33.046 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:32 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:32.750+0000 7fdef7d41000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-01T01:50:33.046 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:32 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:32.909+0000 7fdef7d41000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-01T01:50:33.046 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:32 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:32.990+0000 7fdef7d41000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-01T01:50:33.710 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:33 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:33.297+0000 7fdef7d41000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-01T01:50:33.710 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:33 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:33.385+0000 7fdef7d41000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-01T01:50:33.711 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:33 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:33.475+0000 7fdef7d41000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-01T01:50:33.711 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:33 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:33.711+0000 7fdef7d41000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-01T01:50:34.399 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:34 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:34.141+0000 7fdef7d41000 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-01T01:50:34.684 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:34 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:34.633+0000 7fdef7d41000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-01T01:50:34.778 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: 2024-06-01T01:50:34.778 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: { 2024-06-01T01:50:34.778 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "fsid": "393f964a-1fb9-11ef-bc9b-c7b262605968", 2024-06-01T01:50:34.778 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "health": { 2024-06-01T01:50:34.778 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-06-01T01:50:34.778 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "checks": {}, 2024-06-01T01:50:34.778 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "mutes": [] 2024-06-01T01:50:34.778 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: }, 2024-06-01T01:50:34.778 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-06-01T01:50:34.779 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "quorum": [ 2024-06-01T01:50:34.779 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: 0 2024-06-01T01:50:34.779 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: ], 2024-06-01T01:50:34.779 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "quorum_names": [ 2024-06-01T01:50:34.779 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "a" 2024-06-01T01:50:34.779 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: ], 2024-06-01T01:50:34.779 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "quorum_age": 5, 2024-06-01T01:50:34.779 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "monmap": { 2024-06-01T01:50:34.779 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "epoch": 1, 2024-06-01T01:50:34.779 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "min_mon_release_name": "quincy", 2024-06-01T01:50:34.779 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_mons": 1 2024-06-01T01:50:34.779 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: }, 2024-06-01T01:50:34.779 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "osdmap": { 2024-06-01T01:50:34.780 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "epoch": 1, 2024-06-01T01:50:34.780 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_osds": 0, 2024-06-01T01:50:34.780 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-06-01T01:50:34.780 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-06-01T01:50:34.780 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-06-01T01:50:34.780 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-06-01T01:50:34.780 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-06-01T01:50:34.780 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: }, 2024-06-01T01:50:34.780 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "pgmap": { 2024-06-01T01:50:34.780 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-06-01T01:50:34.780 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-06-01T01:50:34.780 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_pools": 0, 2024-06-01T01:50:34.780 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_objects": 0, 2024-06-01T01:50:34.781 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-06-01T01:50:34.781 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-06-01T01:50:34.781 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-06-01T01:50:34.781 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "bytes_total": 0 2024-06-01T01:50:34.781 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: }, 2024-06-01T01:50:34.781 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "fsmap": { 2024-06-01T01:50:34.781 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "epoch": 1, 2024-06-01T01:50:34.781 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "by_rank": [], 2024-06-01T01:50:34.781 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "up:standby": 0 2024-06-01T01:50:34.781 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: }, 2024-06-01T01:50:34.781 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "mgrmap": { 2024-06-01T01:50:34.781 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "available": false, 2024-06-01T01:50:34.781 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-06-01T01:50:34.781 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "modules": [ 2024-06-01T01:50:34.782 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "iostat", 2024-06-01T01:50:34.782 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "nfs", 2024-06-01T01:50:34.782 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "restful" 2024-06-01T01:50:34.782 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: ], 2024-06-01T01:50:34.782 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "services": {} 2024-06-01T01:50:34.782 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: }, 2024-06-01T01:50:34.782 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "servicemap": { 2024-06-01T01:50:34.782 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "epoch": 1, 2024-06-01T01:50:34.782 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "modified": "2024-06-01T01:50:23.494000+0000", 2024-06-01T01:50:34.782 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "services": {} 2024-06-01T01:50:34.782 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: }, 2024-06-01T01:50:34.782 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "progress_events": {} 2024-06-01T01:50:34.782 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: } 2024-06-01T01:50:34.983 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:34 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:34.727+0000 7fdef7d41000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-01T01:50:35.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:34 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/3453555289' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-01T01:50:35.296 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:35 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:35.030+0000 7fdef7d41000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-01T01:50:35.296 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:35 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:35.118+0000 7fdef7d41000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-01T01:50:35.348 INFO:teuthology.orchestra.run.smithi116.stderr:mgr not available, waiting (2/15)... 2024-06-01T01:50:35.644 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:35 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:35.517+0000 7fdef7d41000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-01T01:50:35.644 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:35 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:35.644+0000 7fdef7d41000 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-01T01:50:35.933 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:35 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:35.734+0000 7fdef7d41000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-01T01:50:35.934 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:35 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:35.847+0000 7fdef7d41000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-01T01:50:35.934 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:35 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:35.934+0000 7fdef7d41000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-01T01:50:36.796 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:36 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:36.468+0000 7fdef7d41000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-01T01:50:37.296 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:36 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:36.822+0000 7fdef7d41000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-01T01:50:37.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:36 smithi116 ceph-mon[25082]: Activating manager daemon y 2024-06-01T01:50:37.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:36 smithi116 ceph-mon[25082]: mgrmap e2: y(active, starting, since 0.00283362s) 2024-06-01T01:50:37.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:36 smithi116 ceph-mon[25082]: from='mgr.14100 172.21.15.116:0/3056029295' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-01T01:50:37.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:36 smithi116 ceph-mon[25082]: from='mgr.14100 172.21.15.116:0/3056029295' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-01T01:50:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:36 smithi116 ceph-mon[25082]: from='mgr.14100 172.21.15.116:0/3056029295' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-01T01:50:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:36 smithi116 ceph-mon[25082]: from='mgr.14100 172.21.15.116:0/3056029295' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T01:50:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:36 smithi116 ceph-mon[25082]: from='mgr.14100 172.21.15.116:0/3056029295' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-01T01:50:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:36 smithi116 ceph-mon[25082]: Manager daemon y is now available 2024-06-01T01:50:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:36 smithi116 ceph-mon[25082]: from='mgr.14100 172.21.15.116:0/3056029295' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T01:50:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:36 smithi116 ceph-mon[25082]: from='mgr.14100 172.21.15.116:0/3056029295' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T01:50:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:36 smithi116 ceph-mon[25082]: from='mgr.14100 172.21.15.116:0/3056029295' entity='mgr.y' 2024-06-01T01:50:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:36 smithi116 ceph-mon[25082]: from='mgr.14100 172.21.15.116:0/3056029295' entity='mgr.y' 2024-06-01T01:50:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:36 smithi116 ceph-mon[25082]: from='mgr.14100 172.21.15.116:0/3056029295' entity='mgr.y' 2024-06-01T01:50:37.818 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: 2024-06-01T01:50:37.818 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: { 2024-06-01T01:50:37.818 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "fsid": "393f964a-1fb9-11ef-bc9b-c7b262605968", 2024-06-01T01:50:37.818 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "health": { 2024-06-01T01:50:37.818 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-06-01T01:50:37.818 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "checks": {}, 2024-06-01T01:50:37.818 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "mutes": [] 2024-06-01T01:50:37.818 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: }, 2024-06-01T01:50:37.818 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-06-01T01:50:37.819 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "quorum": [ 2024-06-01T01:50:37.819 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: 0 2024-06-01T01:50:37.819 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: ], 2024-06-01T01:50:37.819 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "quorum_names": [ 2024-06-01T01:50:37.819 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "a" 2024-06-01T01:50:37.819 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: ], 2024-06-01T01:50:37.819 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "quorum_age": 8, 2024-06-01T01:50:37.819 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "monmap": { 2024-06-01T01:50:37.819 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "epoch": 1, 2024-06-01T01:50:37.819 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "min_mon_release_name": "quincy", 2024-06-01T01:50:37.819 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_mons": 1 2024-06-01T01:50:37.819 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: }, 2024-06-01T01:50:37.819 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "osdmap": { 2024-06-01T01:50:37.820 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "epoch": 1, 2024-06-01T01:50:37.820 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_osds": 0, 2024-06-01T01:50:37.820 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-06-01T01:50:37.820 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-06-01T01:50:37.820 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-06-01T01:50:37.823 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-06-01T01:50:37.823 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-06-01T01:50:37.823 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: }, 2024-06-01T01:50:37.823 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "pgmap": { 2024-06-01T01:50:37.823 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-06-01T01:50:37.823 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-06-01T01:50:37.823 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_pools": 0, 2024-06-01T01:50:37.824 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_objects": 0, 2024-06-01T01:50:37.824 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-06-01T01:50:37.824 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-06-01T01:50:37.824 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-06-01T01:50:37.824 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "bytes_total": 0 2024-06-01T01:50:37.824 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: }, 2024-06-01T01:50:37.824 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "fsmap": { 2024-06-01T01:50:37.824 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "epoch": 1, 2024-06-01T01:50:37.824 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "by_rank": [], 2024-06-01T01:50:37.824 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "up:standby": 0 2024-06-01T01:50:37.824 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: }, 2024-06-01T01:50:37.824 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "mgrmap": { 2024-06-01T01:50:37.824 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "available": false, 2024-06-01T01:50:37.824 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-06-01T01:50:37.825 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "modules": [ 2024-06-01T01:50:37.825 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "iostat", 2024-06-01T01:50:37.825 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "nfs", 2024-06-01T01:50:37.825 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "restful" 2024-06-01T01:50:37.825 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: ], 2024-06-01T01:50:37.825 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "services": {} 2024-06-01T01:50:37.825 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: }, 2024-06-01T01:50:37.825 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "servicemap": { 2024-06-01T01:50:37.825 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "epoch": 1, 2024-06-01T01:50:37.825 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "modified": "2024-06-01T01:50:23.494000+0000", 2024-06-01T01:50:37.825 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "services": {} 2024-06-01T01:50:37.825 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: }, 2024-06-01T01:50:37.825 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "progress_events": {} 2024-06-01T01:50:37.825 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: } 2024-06-01T01:50:38.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:37 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/4035097936' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-01T01:50:38.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:37 smithi116 ceph-mon[25082]: mgrmap e3: y(active, since 1.00572s) 2024-06-01T01:50:38.337 INFO:teuthology.orchestra.run.smithi116.stderr:mgr not available, waiting (3/15)... 2024-06-01T01:50:40.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:39 smithi116 ceph-mon[25082]: mgrmap e4: y(active, since 2s) 2024-06-01T01:50:41.039 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: 2024-06-01T01:50:41.039 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: { 2024-06-01T01:50:41.039 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "fsid": "393f964a-1fb9-11ef-bc9b-c7b262605968", 2024-06-01T01:50:41.039 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "health": { 2024-06-01T01:50:41.039 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-06-01T01:50:41.039 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "checks": {}, 2024-06-01T01:50:41.039 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "mutes": [] 2024-06-01T01:50:41.039 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: }, 2024-06-01T01:50:41.040 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-06-01T01:50:41.040 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "quorum": [ 2024-06-01T01:50:41.040 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: 0 2024-06-01T01:50:41.040 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: ], 2024-06-01T01:50:41.040 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "quorum_names": [ 2024-06-01T01:50:41.040 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "a" 2024-06-01T01:50:41.040 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: ], 2024-06-01T01:50:41.040 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "quorum_age": 12, 2024-06-01T01:50:41.040 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "monmap": { 2024-06-01T01:50:41.040 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "epoch": 1, 2024-06-01T01:50:41.040 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "min_mon_release_name": "quincy", 2024-06-01T01:50:41.040 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_mons": 1 2024-06-01T01:50:41.040 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: }, 2024-06-01T01:50:41.040 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "osdmap": { 2024-06-01T01:50:41.041 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "epoch": 1, 2024-06-01T01:50:41.041 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_osds": 0, 2024-06-01T01:50:41.041 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-06-01T01:50:41.041 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-06-01T01:50:41.041 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-06-01T01:50:41.044 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-06-01T01:50:41.044 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-06-01T01:50:41.044 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: }, 2024-06-01T01:50:41.045 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "pgmap": { 2024-06-01T01:50:41.045 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-06-01T01:50:41.045 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-06-01T01:50:41.045 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_pools": 0, 2024-06-01T01:50:41.045 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_objects": 0, 2024-06-01T01:50:41.045 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-06-01T01:50:41.045 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-06-01T01:50:41.045 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-06-01T01:50:41.045 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "bytes_total": 0 2024-06-01T01:50:41.045 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: }, 2024-06-01T01:50:41.045 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "fsmap": { 2024-06-01T01:50:41.045 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "epoch": 1, 2024-06-01T01:50:41.045 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "by_rank": [], 2024-06-01T01:50:41.046 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "up:standby": 0 2024-06-01T01:50:41.046 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: }, 2024-06-01T01:50:41.046 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "mgrmap": { 2024-06-01T01:50:41.046 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "available": true, 2024-06-01T01:50:41.046 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-06-01T01:50:41.046 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "modules": [ 2024-06-01T01:50:41.046 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "iostat", 2024-06-01T01:50:41.046 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "nfs", 2024-06-01T01:50:41.046 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "restful" 2024-06-01T01:50:41.046 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: ], 2024-06-01T01:50:41.046 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "services": {} 2024-06-01T01:50:41.046 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: }, 2024-06-01T01:50:41.046 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "servicemap": { 2024-06-01T01:50:41.047 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "epoch": 1, 2024-06-01T01:50:41.047 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "modified": "2024-06-01T01:50:23.494000+0000", 2024-06-01T01:50:41.047 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "services": {} 2024-06-01T01:50:41.047 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: }, 2024-06-01T01:50:41.047 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "progress_events": {} 2024-06-01T01:50:41.047 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: } 2024-06-01T01:50:41.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:41 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/3313275208' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-01T01:50:41.630 INFO:teuthology.orchestra.run.smithi116.stderr:mgr is available 2024-06-01T01:50:42.279 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: 2024-06-01T01:50:42.279 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: [global] 2024-06-01T01:50:42.280 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: fsid = 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T01:50:42.280 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: mon_osd_allow_pg_remap = true 2024-06-01T01:50:42.280 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: mon_osd_allow_primary_affinity = true 2024-06-01T01:50:42.280 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: mon_warn_on_no_sortbitwise = false 2024-06-01T01:50:42.280 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: osd_crush_chooseleaf_type = 0 2024-06-01T01:50:42.280 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: 2024-06-01T01:50:42.280 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: [mgr] 2024-06-01T01:50:42.280 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: mgr/telemetry/nag = false 2024-06-01T01:50:42.280 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: 2024-06-01T01:50:42.280 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: [osd] 2024-06-01T01:50:42.280 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: osd_map_max_advance = 10 2024-06-01T01:50:42.280 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: osd_sloppy_crc = true 2024-06-01T01:50:42.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:42 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1831218275' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-01T01:50:42.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:42 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1831218275' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-06-01T01:50:42.737 INFO:teuthology.orchestra.run.smithi116.stderr:Enabling cephadm module... 2024-06-01T01:50:43.702 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:43 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/227044093' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-06-01T01:50:43.702 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:43 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ignoring --setuser ceph since I am not root 2024-06-01T01:50:43.702 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:43 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ignoring --setgroup ceph since I am not root 2024-06-01T01:50:43.702 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:43 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:43.574+0000 7f2decaa3000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-01T01:50:43.702 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:43 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:43.702+0000 7f2decaa3000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-01T01:50:44.000 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:43 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:43.801+0000 7f2decaa3000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-01T01:50:44.296 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:44 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:44.110+0000 7f2decaa3000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-01T01:50:44.296 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:44 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:44.195+0000 7f2decaa3000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-01T01:50:44.694 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:44 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/227044093' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-06-01T01:50:44.694 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:44 smithi116 ceph-mon[25082]: mgrmap e5: y(active, since 6s) 2024-06-01T01:50:44.694 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: { 2024-06-01T01:50:44.695 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "epoch": 5, 2024-06-01T01:50:44.695 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "available": true, 2024-06-01T01:50:44.695 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "active_name": "y", 2024-06-01T01:50:44.695 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_standby": 0 2024-06-01T01:50:44.695 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: } 2024-06-01T01:50:45.381 INFO:teuthology.orchestra.run.smithi116.stderr:Waiting for the mgr to restart... 2024-06-01T01:50:45.381 INFO:teuthology.orchestra.run.smithi116.stderr:Waiting for mgr epoch 5... 2024-06-01T01:50:45.600 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:45 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/868696609' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-01T01:50:45.600 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:45 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:45.350+0000 7f2decaa3000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-01T01:50:45.600 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:45 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:45.519+0000 7f2decaa3000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-01T01:50:45.600 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:45 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:45.600+0000 7f2decaa3000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-01T01:50:46.296 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:45 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:45.918+0000 7f2decaa3000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-01T01:50:46.296 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:46 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:46.007+0000 7f2decaa3000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-01T01:50:46.296 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:46 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:46.098+0000 7f2decaa3000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-01T01:50:46.670 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:46 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:46.349+0000 7f2decaa3000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-01T01:50:47.046 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:46 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:46.790+0000 7f2decaa3000 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-01T01:50:47.386 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:47.292+0000 7f2decaa3000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-01T01:50:47.386 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:47.387+0000 7f2decaa3000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-01T01:50:48.046 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:47.682+0000 7f2decaa3000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-01T01:50:48.046 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:47.772+0000 7f2decaa3000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-01T01:50:48.500 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:48 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:48.171+0000 7f2decaa3000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-01T01:50:48.501 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:48 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:48.300+0000 7f2decaa3000 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-01T01:50:48.501 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:48 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:48.391+0000 7f2decaa3000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-01T01:50:48.501 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:48 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:48.501+0000 7f2decaa3000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-01T01:50:48.796 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:48 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:48.588+0000 7f2decaa3000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-01T01:50:49.478 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:49 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:49.124+0000 7f2decaa3000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-01T01:50:49.796 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:49 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:50:49.479+0000 7f2decaa3000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-01T01:50:50.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:50 smithi116 ceph-mon[25082]: Active manager daemon y restarted 2024-06-01T01:50:50.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:50 smithi116 ceph-mon[25082]: Activating manager daemon y 2024-06-01T01:50:50.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:50 smithi116 ceph-mon[25082]: osdmap e2: 0 total, 0 up, 0 in 2024-06-01T01:50:50.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:50 smithi116 ceph-mon[25082]: mgrmap e6: y(active, starting, since 0.507872s) 2024-06-01T01:50:50.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:50 smithi116 ceph-mon[25082]: from='mgr.14120 172.21.15.116:0/986257448' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T01:50:50.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:50 smithi116 ceph-mon[25082]: from='mgr.14120 172.21.15.116:0/986257448' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-01T01:50:50.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:50 smithi116 ceph-mon[25082]: from='mgr.14120 172.21.15.116:0/986257448' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-01T01:50:50.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:50 smithi116 ceph-mon[25082]: from='mgr.14120 172.21.15.116:0/986257448' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-01T01:50:50.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:50 smithi116 ceph-mon[25082]: from='mgr.14120 172.21.15.116:0/986257448' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-01T01:50:50.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:50 smithi116 ceph-mon[25082]: Manager daemon y is now available 2024-06-01T01:50:50.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:50 smithi116 ceph-mon[25082]: from='mgr.14120 172.21.15.116:0/986257448' entity='mgr.y' 2024-06-01T01:50:50.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:50 smithi116 ceph-mon[25082]: from='mgr.14120 172.21.15.116:0/986257448' entity='mgr.y' 2024-06-01T01:50:50.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:50 smithi116 ceph-mon[25082]: from='mgr.14120 172.21.15.116:0/986257448' entity='mgr.y' 2024-06-01T01:50:50.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:50 smithi116 ceph-mon[25082]: from='mgr.14120 172.21.15.116:0/986257448' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:50:50.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:50 smithi116 ceph-mon[25082]: from='mgr.14120 172.21.15.116:0/986257448' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:50:50.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:50 smithi116 ceph-mon[25082]: from='mgr.14120 172.21.15.116:0/986257448' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:50:50.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:50 smithi116 ceph-mon[25082]: from='mgr.14120 172.21.15.116:0/986257448' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T01:50:50.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:50 smithi116 ceph-mon[25082]: from='mgr.14120 172.21.15.116:0/986257448' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T01:50:51.001 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: { 2024-06-01T01:50:51.002 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "mgrmap_epoch": 7, 2024-06-01T01:50:51.002 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "initialized": true 2024-06-01T01:50:51.002 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: } 2024-06-01T01:50:51.264 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:51 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: [01/Jun/2024:01:50:51] ENGINE Bus STARTING 2024-06-01T01:50:51.264 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:51 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: [01/Jun/2024:01:50:51] ENGINE Serving on https://172.21.15.116:7150 2024-06-01T01:50:51.546 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:51 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: [01/Jun/2024:01:50:51] ENGINE Bus STARTED 2024-06-01T01:50:51.660 INFO:teuthology.orchestra.run.smithi116.stderr:mgr epoch 5 is available 2024-06-01T01:50:51.661 INFO:teuthology.orchestra.run.smithi116.stderr:Setting orchestrator backend to cephadm... 2024-06-01T01:50:52.284 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:51 smithi116 ceph-mon[25082]: mgrmap e7: y(active, since 1.51027s) 2024-06-01T01:50:52.285 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:51 smithi116 ceph-mon[25082]: [01/Jun/2024:01:50:51] ENGINE Bus STARTING 2024-06-01T01:50:52.285 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:51 smithi116 ceph-mon[25082]: [01/Jun/2024:01:50:51] ENGINE Serving on https://172.21.15.116:7150 2024-06-01T01:50:52.285 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:51 smithi116 ceph-mon[25082]: [01/Jun/2024:01:50:51] ENGINE Bus STARTED 2024-06-01T01:50:52.285 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:51 smithi116 ceph-mon[25082]: from='mgr.14120 172.21.15.116:0/986257448' entity='mgr.y' 2024-06-01T01:50:52.285 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:51 smithi116 ceph-mon[25082]: from='mgr.14120 172.21.15.116:0/986257448' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:50:53.427 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:53 smithi116 ceph-mon[25082]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:50:53.427 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:53 smithi116 ceph-mon[25082]: from='mgr.14120 172.21.15.116:0/986257448' entity='mgr.y' 2024-06-01T01:50:53.427 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:53 smithi116 ceph-mon[25082]: mgrmap e8: y(active, since 2s) 2024-06-01T01:50:53.427 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:53 smithi116 ceph-mon[25082]: from='mgr.14120 172.21.15.116:0/986257448' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:50:53.428 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: value unchanged 2024-06-01T01:50:54.021 INFO:teuthology.orchestra.run.smithi116.stderr:Generating ssh key... 2024-06-01T01:50:54.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:54 smithi116 ceph-mon[25082]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:50:55.118 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:54 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: Generating public/private rsa key pair. 2024-06-01T01:50:55.119 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:54 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: Your identification has been saved in /tmp/tmp88f648ax/key. 2024-06-01T01:50:55.119 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:54 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: Your public key has been saved in /tmp/tmp88f648ax/key.pub. 2024-06-01T01:50:55.119 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:54 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: The key fingerprint is: 2024-06-01T01:50:55.119 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:54 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: SHA256:67LXNVti/f8m2XidO1KgeBl5/9vlxVtBhpn6eqvdeiw ceph-393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T01:50:55.119 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:54 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: The key's randomart image is: 2024-06-01T01:50:55.119 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:54 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: +---[RSA 3072]----+ 2024-06-01T01:50:55.119 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:54 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: | | 2024-06-01T01:50:55.119 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:54 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: | + | 2024-06-01T01:50:55.119 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:54 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: | .+ o | 2024-06-01T01:50:55.119 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:54 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: | o.oo | 2024-06-01T01:50:55.119 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:54 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: | S ..=.o. | 2024-06-01T01:50:55.119 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:54 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: | o +* o+.| 2024-06-01T01:50:55.119 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:54 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: | ...o *o=O| 2024-06-01T01:50:55.120 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:54 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: | ... . +E=*&| 2024-06-01T01:50:55.120 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:54 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: | .+. oo+*O@| 2024-06-01T01:50:55.120 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:50:54 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: +----[SHA256]-----+ 2024-06-01T01:50:55.970 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:55 smithi116 ceph-mon[25082]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:50:55.970 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:55 smithi116 ceph-mon[25082]: Generating ssh key... 2024-06-01T01:50:55.970 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:55 smithi116 ceph-mon[25082]: from='mgr.14120 172.21.15.116:0/986257448' entity='mgr.y' 2024-06-01T01:50:55.970 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:55 smithi116 ceph-mon[25082]: from='mgr.14120 172.21.15.116:0/986257448' entity='mgr.y' 2024-06-01T01:50:55.970 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFgo+dXxnZD2HRt4Bygslka4l4IkjizTmu0AdmRmQZfd7mfRw4rN6auOhUrJjO3Rx43Cf7w6gEZ8cRZxVbvpqPYSnhxpo41dyh9slc5E06vt8yfuvnbXC8bbWfw0B7q0pce6YWraMrr14pSA6BQnc1cmVECxYaWrL1d7q6EP9RFZfXLL6/q0fOTY1barXsB17PqGOiJL8hiWw57ei2tW1/frGbaDSqZBt3b2WaNqy2luHQDgeu8ltmVl7j8xFPv9lrrPvHQE6z29fDnLiAJLR+2kCLsmeeZDQVLcySs8fIltIiRioE2/L6gW7/84lRhgC34eaPAZwZD7FY22lwQ7yz40KAlV/L/dtIXGJ9v0HJ1cROLKWEMU8XEB7X3DHVtNVKEDlaDMVQ0yuv9TG5D3LuyB17Q9Z4CABuc4IhuXAJKN03GZxCdtFl/i/qRvLqfQ84OSZPWgSdeOuGNt3hduSaIl4zVyY/TEcVGE6ki0vkh39ZCIzV3YSkWSnENvH4668= ceph-393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T01:50:56.594 INFO:teuthology.orchestra.run.smithi116.stderr:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-06-01T01:50:56.594 INFO:teuthology.orchestra.run.smithi116.stderr:Adding key to root@localhost authorized_keys... 2024-06-01T01:50:56.595 INFO:teuthology.orchestra.run.smithi116.stderr:Adding host smithi116... 2024-06-01T01:50:57.217 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:56 smithi116 ceph-mon[25082]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:50:58.063 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:57 smithi116 ceph-mon[25082]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi116", "addr": "172.21.15.116", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:50:58.072 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: Added host 'smithi116' with addr '172.21.15.116' 2024-06-01T01:50:58.651 INFO:teuthology.orchestra.run.smithi116.stderr:Deploying unmanaged mon service... 2024-06-01T01:50:59.248 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:59 smithi116 ceph-mon[25082]: Deploying cephadm binary to smithi116 2024-06-01T01:50:59.249 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:59 smithi116 ceph-mon[25082]: from='mgr.14120 172.21.15.116:0/986257448' entity='mgr.y' 2024-06-01T01:50:59.249 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:59 smithi116 ceph-mon[25082]: Added host smithi116 2024-06-01T01:50:59.249 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:50:59 smithi116 ceph-mon[25082]: from='mgr.14120 172.21.15.116:0/986257448' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:50:59.249 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: Scheduled mon update... 2024-06-01T01:51:00.068 INFO:teuthology.orchestra.run.smithi116.stderr:Deploying unmanaged mgr service... 2024-06-01T01:51:00.317 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:00 smithi116 ceph-mon[25082]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:51:00.317 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:00 smithi116 ceph-mon[25082]: Saving service mon spec with placement count:5 2024-06-01T01:51:00.318 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:00 smithi116 ceph-mon[25082]: from='mgr.14120 172.21.15.116:0/986257448' entity='mgr.y' 2024-06-01T01:51:01.774 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: Scheduled mgr update... 2024-06-01T01:51:02.366 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:02 smithi116 ceph-mon[25082]: from='mgr.14120 172.21.15.116:0/986257448' entity='mgr.y' 2024-06-01T01:51:02.366 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:02 smithi116 ceph-mon[25082]: from='mgr.14120 172.21.15.116:0/986257448' entity='mgr.y' 2024-06-01T01:51:02.366 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:02 smithi116 ceph-mon[25082]: from='mgr.14120 172.21.15.116:0/986257448' entity='mgr.y' 2024-06-01T01:51:03.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:03 smithi116 ceph-mon[25082]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:51:03.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:03 smithi116 ceph-mon[25082]: Saving service mgr spec with placement count:2 2024-06-01T01:51:03.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:03 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/4139324627' entity='client.admin' 2024-06-01T01:51:05.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:04 smithi116 ceph-mon[25082]: from='mgr.14120 172.21.15.116:0/986257448' entity='mgr.y' 2024-06-01T01:51:05.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:04 smithi116 ceph-mon[25082]: from='mgr.14120 172.21.15.116:0/986257448' entity='mgr.y' 2024-06-01T01:51:05.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:04 smithi116 ceph-mon[25082]: from='mgr.14120 172.21.15.116:0/986257448' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi116", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:51:05.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:04 smithi116 ceph-mon[25082]: from='mgr.14120 172.21.15.116:0/986257448' entity='mgr.y' 2024-06-01T01:51:05.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:04 smithi116 ceph-mon[25082]: from='mgr.14120 172.21.15.116:0/986257448' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T01:51:05.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:04 smithi116 ceph-mon[25082]: from='mgr.14120 172.21.15.116:0/986257448' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]': finished 2024-06-01T01:51:05.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:04 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/2364236095' entity='client.admin' 2024-06-01T01:51:05.322 INFO:teuthology.orchestra.run.smithi116.stderr:Enabling the dashboard module... 2024-06-01T01:51:06.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:05 smithi116 ceph-mon[25082]: Deploying daemon agent.smithi116 on smithi116 2024-06-01T01:51:07.124 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:06 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1040021611' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-06-01T01:51:07.124 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:06 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ignoring --setuser ceph since I am not root 2024-06-01T01:51:07.124 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:06 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ignoring --setgroup ceph since I am not root 2024-06-01T01:51:07.124 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:06 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:51:06.995+0000 7f3f7fc84000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-01T01:51:07.124 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:07 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:51:07.124+0000 7f3f7fc84000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-01T01:51:07.482 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:07 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:51:07.222+0000 7f3f7fc84000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-01T01:51:07.741 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:07 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:51:07.531+0000 7f3f7fc84000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-01T01:51:07.741 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:07 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:51:07.613+0000 7f3f7fc84000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-01T01:51:08.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:07 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1040021611' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-06-01T01:51:08.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:07 smithi116 ceph-mon[25082]: mgrmap e9: y(active, since 17s) 2024-06-01T01:51:08.218 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: { 2024-06-01T01:51:08.218 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "epoch": 9, 2024-06-01T01:51:08.218 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "available": true, 2024-06-01T01:51:08.218 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "active_name": "y", 2024-06-01T01:51:08.218 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "num_standby": 0 2024-06-01T01:51:08.218 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: } 2024-06-01T01:51:08.831 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:08 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:51:08.755+0000 7f3f7fc84000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-01T01:51:08.874 INFO:teuthology.orchestra.run.smithi116.stderr:Waiting for the mgr to restart... 2024-06-01T01:51:08.874 INFO:teuthology.orchestra.run.smithi116.stderr:Waiting for mgr epoch 9... 2024-06-01T01:51:09.117 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:08 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1989610564' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-01T01:51:09.117 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:08 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:51:08.927+0000 7f3f7fc84000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-01T01:51:09.117 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:09 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:51:09.010+0000 7f3f7fc84000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-01T01:51:09.423 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:09 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:51:09.334+0000 7f3f7fc84000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-01T01:51:09.423 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:09 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:51:09.423+0000 7f3f7fc84000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-01T01:51:09.754 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:09 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:51:09.517+0000 7f3f7fc84000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-01T01:51:09.754 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:09 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:51:09.755+0000 7f3f7fc84000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-01T01:51:10.546 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:10 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:51:10.193+0000 7f3f7fc84000 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-01T01:51:11.046 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:10 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:51:10.672+0000 7f3f7fc84000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-01T01:51:11.046 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:10 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:51:10.765+0000 7f3f7fc84000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-01T01:51:11.538 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:51:11.053+0000 7f3f7fc84000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-01T01:51:11.538 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:51:11.142+0000 7f3f7fc84000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-01T01:51:11.796 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:51:11.538+0000 7f3f7fc84000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-01T01:51:11.796 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:51:11.669+0000 7f3f7fc84000 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-01T01:51:11.796 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:51:11.760+0000 7f3f7fc84000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-01T01:51:12.296 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:51:11.873+0000 7f3f7fc84000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-01T01:51:12.296 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:51:11.962+0000 7f3f7fc84000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-01T01:51:12.796 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:12 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:51:12.503+0000 7f3f7fc84000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-01T01:51:13.296 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:12 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:51:12.857+0000 7f3f7fc84000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-01T01:51:13.796 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:13 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: [01/Jun/2024:01:51:13] ENGINE Bus STARTING 2024-06-01T01:51:13.796 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:13 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: [01/Jun/2024:01:51:13] ENGINE Serving on https://172.21.15.116:7150 2024-06-01T01:51:13.796 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:13 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: [01/Jun/2024:01:51:13] ENGINE Bus STARTED 2024-06-01T01:51:13.922 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: { 2024-06-01T01:51:13.922 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "mgrmap_epoch": 11, 2024-06-01T01:51:13.922 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: "initialized": true 2024-06-01T01:51:13.922 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: } 2024-06-01T01:51:14.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:13 smithi116 ceph-mon[25082]: Active manager daemon y restarted 2024-06-01T01:51:14.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:13 smithi116 ceph-mon[25082]: Activating manager daemon y 2024-06-01T01:51:14.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:13 smithi116 ceph-mon[25082]: osdmap e3: 0 total, 0 up, 0 in 2024-06-01T01:51:14.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:13 smithi116 ceph-mon[25082]: mgrmap e10: y(active, starting, since 0.0551555s) 2024-06-01T01:51:14.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:13 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T01:51:14.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:13 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-01T01:51:14.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:13 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-01T01:51:14.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:13 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-01T01:51:14.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:13 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-01T01:51:14.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:13 smithi116 ceph-mon[25082]: Manager daemon y is now available 2024-06-01T01:51:14.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:13 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:14.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:13 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:51:14.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:13 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:51:14.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:13 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T01:51:14.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:13 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T01:51:14.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:13 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:14.593 INFO:teuthology.orchestra.run.smithi116.stderr:mgr epoch 9 is available 2024-06-01T01:51:14.593 INFO:teuthology.orchestra.run.smithi116.stderr:Generating a dashboard self-signed certificate... 2024-06-01T01:51:14.994 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:14 smithi116 ceph-mon[25082]: [01/Jun/2024:01:51:13] ENGINE Bus STARTING 2024-06-01T01:51:14.994 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:14 smithi116 ceph-mon[25082]: [01/Jun/2024:01:51:13] ENGINE Serving on https://172.21.15.116:7150 2024-06-01T01:51:14.994 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:14 smithi116 ceph-mon[25082]: [01/Jun/2024:01:51:13] ENGINE Bus STARTED 2024-06-01T01:51:14.994 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:14 smithi116 ceph-mon[25082]: mgrmap e11: y(active, since 1.05748s) 2024-06-01T01:51:15.330 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: Self-signed certificate created 2024-06-01T01:51:16.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:15 smithi116 ceph-mon[25082]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-01T01:51:16.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:15 smithi116 ceph-mon[25082]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-01T01:51:16.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:15 smithi116 ceph-mon[25082]: mgrmap e12: y(active, since 2s) 2024-06-01T01:51:16.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:15 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:16.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:15 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi116", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:51:16.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:15 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:16.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:15 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T01:51:16.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:15 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:16.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:15 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:16.050 INFO:teuthology.orchestra.run.smithi116.stderr:Creating initial admin user... 2024-06-01T01:51:16.946 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: {"username": "admin", "password": "$2b$12$bOy92NlYl8OOxBjI/r5otOImpExEMVwi8TrcA5R7Tjr1Ns.QDDZs2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1717206676, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-06-01T01:51:17.237 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:16 smithi116 ceph-mon[25082]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:51:17.237 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:16 smithi116 ceph-mon[25082]: Deploying daemon agent.smithi116 on smithi116 2024-06-01T01:51:17.549 INFO:teuthology.orchestra.run.smithi116.stderr:Fetching dashboard port number... 2024-06-01T01:51:18.208 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:17 smithi116 ceph-mon[25082]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:51:18.208 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:17 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:18.209 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: 8443 2024-06-01T01:51:18.849 INFO:teuthology.orchestra.run.smithi116.stderr:systemctl: disabled 2024-06-01T01:51:18.856 INFO:teuthology.orchestra.run.smithi116.stderr:systemctl: inactive 2024-06-01T01:51:18.858 INFO:teuthology.orchestra.run.smithi116.stderr:firewalld.service is not enabled 2024-06-01T01:51:18.858 INFO:teuthology.orchestra.run.smithi116.stderr:Not possible to open ports <[8443]>. firewalld.service is not available 2024-06-01T01:51:18.859 INFO:teuthology.orchestra.run.smithi116.stderr:Ceph Dashboard is now available at: 2024-06-01T01:51:18.859 INFO:teuthology.orchestra.run.smithi116.stderr: 2024-06-01T01:51:18.859 INFO:teuthology.orchestra.run.smithi116.stderr: URL: https://smithi116.front.sepia.ceph.com:8443/ 2024-06-01T01:51:18.860 INFO:teuthology.orchestra.run.smithi116.stderr: User: admin 2024-06-01T01:51:18.860 INFO:teuthology.orchestra.run.smithi116.stderr: Password: 8ny74zc150 2024-06-01T01:51:18.860 INFO:teuthology.orchestra.run.smithi116.stderr: 2024-06-01T01:51:18.860 INFO:teuthology.orchestra.run.smithi116.stderr:Enabling autotune for osd_memory_target 2024-06-01T01:51:19.090 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:18 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/4022315404' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-06-01T01:51:20.035 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:19 smithi116 ceph-mon[25082]: mgrmap e13: y(active, since 6s) 2024-06-01T01:51:20.762 INFO:teuthology.orchestra.run.smithi116.stderr:/usr/bin/ceph: set mgr/dashboard/cluster/status 2024-06-01T01:51:21.444 INFO:teuthology.orchestra.run.smithi116.stderr:You can access the Ceph CLI with: 2024-06-01T01:51:21.444 INFO:teuthology.orchestra.run.smithi116.stderr: 2024-06-01T01:51:21.444 INFO:teuthology.orchestra.run.smithi116.stderr: sudo /home/ubuntu/cephtest/cephadm shell --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-06-01T01:51:21.445 INFO:teuthology.orchestra.run.smithi116.stderr: 2024-06-01T01:51:21.445 INFO:teuthology.orchestra.run.smithi116.stderr:Please consider enabling telemetry to help improve Ceph: 2024-06-01T01:51:21.445 INFO:teuthology.orchestra.run.smithi116.stderr: 2024-06-01T01:51:21.445 INFO:teuthology.orchestra.run.smithi116.stderr: ceph telemetry on 2024-06-01T01:51:21.445 INFO:teuthology.orchestra.run.smithi116.stderr: 2024-06-01T01:51:21.445 INFO:teuthology.orchestra.run.smithi116.stderr:For more information see: 2024-06-01T01:51:21.445 INFO:teuthology.orchestra.run.smithi116.stderr: 2024-06-01T01:51:21.445 INFO:teuthology.orchestra.run.smithi116.stderr: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-06-01T01:51:21.445 INFO:teuthology.orchestra.run.smithi116.stderr: 2024-06-01T01:51:21.445 INFO:teuthology.orchestra.run.smithi116.stderr:Bootstrap complete. 2024-06-01T01:51:21.479 INFO:tasks.cephadm:Fetching config... 2024-06-01T01:51:21.479 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-06-01T01:51:21.479 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-06-01T01:51:21.496 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-06-01T01:51:21.496 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-06-01T01:51:21.496 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-06-01T01:51:21.552 INFO:tasks.cephadm:Fetching mon keyring... 2024-06-01T01:51:21.552 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-06-01T01:51:21.552 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/mon.a/keyring of=/dev/stdout 2024-06-01T01:51:21.676 INFO:tasks.cephadm:Fetching pub ssh key... 2024-06-01T01:51:21.677 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-06-01T01:51:21.677 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-06-01T01:51:21.734 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-06-01T01:51:21.735 DEBUG:teuthology.orchestra.run.smithi116:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFgo+dXxnZD2HRt4Bygslka4l4IkjizTmu0AdmRmQZfd7mfRw4rN6auOhUrJjO3Rx43Cf7w6gEZ8cRZxVbvpqPYSnhxpo41dyh9slc5E06vt8yfuvnbXC8bbWfw0B7q0pce6YWraMrr14pSA6BQnc1cmVECxYaWrL1d7q6EP9RFZfXLL6/q0fOTY1barXsB17PqGOiJL8hiWw57ei2tW1/frGbaDSqZBt3b2WaNqy2luHQDgeu8ltmVl7j8xFPv9lrrPvHQE6z29fDnLiAJLR+2kCLsmeeZDQVLcySs8fIltIiRioE2/L6gW7/84lRhgC34eaPAZwZD7FY22lwQ7yz40KAlV/L/dtIXGJ9v0HJ1cROLKWEMU8XEB7X3DHVtNVKEDlaDMVQ0yuv9TG5D3LuyB17Q9Z4CABuc4IhuXAJKN03GZxCdtFl/i/qRvLqfQ84OSZPWgSdeOuGNt3hduSaIl4zVyY/TEcVGE6ki0vkh39ZCIzV3YSkWSnENvH4668= ceph-393f964a-1fb9-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-01T01:51:21.877 INFO:teuthology.orchestra.run.smithi116.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFgo+dXxnZD2HRt4Bygslka4l4IkjizTmu0AdmRmQZfd7mfRw4rN6auOhUrJjO3Rx43Cf7w6gEZ8cRZxVbvpqPYSnhxpo41dyh9slc5E06vt8yfuvnbXC8bbWfw0B7q0pce6YWraMrr14pSA6BQnc1cmVECxYaWrL1d7q6EP9RFZfXLL6/q0fOTY1barXsB17PqGOiJL8hiWw57ei2tW1/frGbaDSqZBt3b2WaNqy2luHQDgeu8ltmVl7j8xFPv9lrrPvHQE6z29fDnLiAJLR+2kCLsmeeZDQVLcySs8fIltIiRioE2/L6gW7/84lRhgC34eaPAZwZD7FY22lwQ7yz40KAlV/L/dtIXGJ9v0HJ1cROLKWEMU8XEB7X3DHVtNVKEDlaDMVQ0yuv9TG5D3LuyB17Q9Z4CABuc4IhuXAJKN03GZxCdtFl/i/qRvLqfQ84OSZPWgSdeOuGNt3hduSaIl4zVyY/TEcVGE6ki0vkh39ZCIzV3YSkWSnENvH4668= ceph-393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T01:51:21.890 DEBUG:teuthology.orchestra.run.smithi161:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFgo+dXxnZD2HRt4Bygslka4l4IkjizTmu0AdmRmQZfd7mfRw4rN6auOhUrJjO3Rx43Cf7w6gEZ8cRZxVbvpqPYSnhxpo41dyh9slc5E06vt8yfuvnbXC8bbWfw0B7q0pce6YWraMrr14pSA6BQnc1cmVECxYaWrL1d7q6EP9RFZfXLL6/q0fOTY1barXsB17PqGOiJL8hiWw57ei2tW1/frGbaDSqZBt3b2WaNqy2luHQDgeu8ltmVl7j8xFPv9lrrPvHQE6z29fDnLiAJLR+2kCLsmeeZDQVLcySs8fIltIiRioE2/L6gW7/84lRhgC34eaPAZwZD7FY22lwQ7yz40KAlV/L/dtIXGJ9v0HJ1cROLKWEMU8XEB7X3DHVtNVKEDlaDMVQ0yuv9TG5D3LuyB17Q9Z4CABuc4IhuXAJKN03GZxCdtFl/i/qRvLqfQ84OSZPWgSdeOuGNt3hduSaIl4zVyY/TEcVGE6ki0vkh39ZCIzV3YSkWSnENvH4668= ceph-393f964a-1fb9-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-01T01:51:21.932 INFO:teuthology.orchestra.run.smithi161.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFgo+dXxnZD2HRt4Bygslka4l4IkjizTmu0AdmRmQZfd7mfRw4rN6auOhUrJjO3Rx43Cf7w6gEZ8cRZxVbvpqPYSnhxpo41dyh9slc5E06vt8yfuvnbXC8bbWfw0B7q0pce6YWraMrr14pSA6BQnc1cmVECxYaWrL1d7q6EP9RFZfXLL6/q0fOTY1barXsB17PqGOiJL8hiWw57ei2tW1/frGbaDSqZBt3b2WaNqy2luHQDgeu8ltmVl7j8xFPv9lrrPvHQE6z29fDnLiAJLR+2kCLsmeeZDQVLcySs8fIltIiRioE2/L6gW7/84lRhgC34eaPAZwZD7FY22lwQ7yz40KAlV/L/dtIXGJ9v0HJ1cROLKWEMU8XEB7X3DHVtNVKEDlaDMVQ0yuv9TG5D3LuyB17Q9Z4CABuc4IhuXAJKN03GZxCdtFl/i/qRvLqfQ84OSZPWgSdeOuGNt3hduSaIl4zVyY/TEcVGE6ki0vkh39ZCIzV3YSkWSnENvH4668= ceph-393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T01:51:21.945 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-06-01T01:51:21.973 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:21 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/3592176357' entity='client.admin' 2024-06-01T01:51:23.012 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:22 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:51:22] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:51:23.670 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:23 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:23.670 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:23 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:51:23.670 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:23 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:23.670 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:23 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:51:23.670 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:23 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:23.670 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:23 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:23.670 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:23 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:25.090 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-06-01T01:51:25.091 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-06-01T01:51:25.446 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:25 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/4160576505' entity='client.admin' 2024-06-01T01:51:25.446 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:25 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:51:25.446 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:25 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:28.474 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:28 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:51:28] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:51:29.046 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:28 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:51:28] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:51:29.412 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:29 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:29.413 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:29 smithi116 ceph-mon[25082]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:51:29.413 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:29 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:29.413 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:29 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:29.413 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:29 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:29.413 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:29 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:51:29.413 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:29 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:51:29.413 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:29 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:51:29.413 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:29 smithi116 ceph-mon[25082]: Updating smithi116:/etc/ceph/ceph.conf 2024-06-01T01:51:29.413 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:29 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:29.413 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:29 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:29.435 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi161 2024-06-01T01:51:29.435 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-06-01T01:51:29.435 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/etc/ceph/ceph.conf 2024-06-01T01:51:29.454 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-06-01T01:51:29.455 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-01T01:51:29.513 INFO:tasks.cephadm:Adding host smithi161 to orchestrator... 2024-06-01T01:51:29.513 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph orch host add smithi161 2024-06-01T01:51:30.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:30 smithi116 ceph-mon[25082]: Updating smithi116:/etc/ceph/ceph.client.admin.keyring 2024-06-01T01:51:32.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:31 smithi116 ceph-mon[25082]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi161", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:51:32.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:31 smithi116 ceph-mon[25082]: Deploying cephadm binary to smithi161 2024-06-01T01:51:32.294 INFO:teuthology.orchestra.run.smithi116.stdout:Added host 'smithi161' with addr '172.21.15.161' 2024-06-01T01:51:32.881 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-06-01T01:51:33.519 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:33 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:33.520 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:33 smithi116 ceph-mon[25082]: Added host smithi161 2024-06-01T01:51:33.520 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:33 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:51:33.520 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:33 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:51:33.520 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:33 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:51:33.520 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:33 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:33.520 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:33 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:33.520 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:33 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:33.520 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:33 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:51:33.520 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:33 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:33.520 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:33 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T01:51:33.520 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:33 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]': finished 2024-06-01T01:51:34.409 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:34 smithi116 ceph-mon[25082]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:51:34.409 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:34 smithi116 ceph-mon[25082]: Deploying daemon agent.smithi161 on smithi161 2024-06-01T01:51:34.410 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:34 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:34.410 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:34 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:51:34.410 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:34 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:51:34.410 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:34 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:51:34.660 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:34 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:51:34] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:51:34.855 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-06-01T01:51:34.856 INFO:teuthology.orchestra.run.smithi116.stdout:[{"addr": "172.21.15.116", "hostname": "smithi116", "labels": [], "status": ""}, {"addr": "172.21.15.161", "hostname": "smithi161", "labels": [], "status": ""}] 2024-06-01T01:51:35.394 INFO:tasks.cephadm:Setting crush tunables to default 2024-06-01T01:51:35.394 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph osd crush tunables default 2024-06-01T01:51:35.796 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:35 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:51:35] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:51:35.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:35 smithi116 ceph-mon[25082]: Updating smithi161:/etc/ceph/ceph.conf 2024-06-01T01:51:35.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:35 smithi116 ceph-mon[25082]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-06-01T01:51:35.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:35 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:35.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:35 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:35.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:35 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:51:35.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:35 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:51:35.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:35 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:51:35.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:35 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:35.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:35 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:35.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:35 smithi116 ceph-mon[25082]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-01T01:51:36.636 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:36 smithi116 ceph-mon[25082]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:51:36.636 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:36 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:36.636 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:36 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:37.296 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:37 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:51:37] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:51:38.184 INFO:teuthology.orchestra.run.smithi116.stderr:adjusted tunables profile to default 2024-06-01T01:51:38.508 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:38 smithi116 ceph-mon[25082]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:51:38.508 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:38 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:38.508 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:38 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:38.508 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:38 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1266072926' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-01T01:51:38.508 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:38 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:38.778 INFO:tasks.cephadm:Adding mon.a on smithi116 2024-06-01T01:51:38.778 INFO:tasks.cephadm:Adding mon.c on smithi116 2024-06-01T01:51:38.778 INFO:tasks.cephadm:Adding mon.b on smithi161 2024-06-01T01:51:38.778 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph orch apply mon '3;smithi116:172.21.15.116=a;smithi116:[v2:172.21.15.116:3301,v1:172.21.15.116:6790]=c;smithi161:172.21.15.161=b' 2024-06-01T01:51:39.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:39 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1266072926' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-01T01:51:39.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:39 smithi116 ceph-mon[25082]: osdmap e4: 0 total, 0 up, 0 in 2024-06-01T01:51:40.392 INFO:teuthology.orchestra.run.smithi161.stdout:Scheduled mon update... 2024-06-01T01:51:40.471 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:40 smithi116 ceph-mon[25082]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:51:40.858 DEBUG:teuthology.orchestra.run.smithi116:mon.c> sudo journalctl -f -n 0 -u ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mon.c.service 2024-06-01T01:51:40.861 DEBUG:teuthology.orchestra.run.smithi161:mon.b> sudo journalctl -f -n 0 -u ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mon.b.service 2024-06-01T01:51:40.863 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-01T01:51:40.863 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-01T01:51:41.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:41 smithi116 ceph-mon[25082]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi116:172.21.15.116=a;smithi116:[v2:172.21.15.116:3301,v1:172.21.15.116:6790]=c;smithi161:172.21.15.161=b", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:51:41.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:41 smithi116 ceph-mon[25082]: Saving service mon spec with placement smithi116:172.21.15.116=a;smithi116:[v2:172.21.15.116:3301,v1:172.21.15.116:6790]=c;smithi161:172.21.15.161=b;count:3 2024-06-01T01:51:41.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:41 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:41.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:41 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:51:41.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:41 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:51:41.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:41 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:51:41.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:41 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:41.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:41 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T01:51:41.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:41 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:51:41.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:41 smithi116 ceph-mon[25082]: Deploying daemon mon.c on smithi116 2024-06-01T01:51:42.742 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-06-01T01:51:42.742 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":1,"fsid":"393f964a-1fb9-11ef-bc9b-c7b262605968","modified":"2024-06-01T01:50:20.260187Z","created":"2024-06-01T01:50:20.260187Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:3300","nonce":0},{"type":"v1","addr":"172.21.15.116:6789","nonce":0}]},"addr":"172.21.15.116:6789/0","public_addr":"172.21.15.116:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-01T01:51:42.744 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 1 2024-06-01T01:51:42.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:42 smithi116 ceph-mon[25082]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:51:43.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:43 smithi116 ceph-mon[25082]: from='client.? 172.21.15.161:0/4186891177' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-01T01:51:44.298 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-01T01:51:44.298 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-01T01:51:44.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:44 smithi116 ceph-mon[25082]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:51:44.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:44 smithi116 ceph-mon[29958]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete,mon_cluster_log_to_file 2024-06-01T01:51:44.884 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:44 smithi116 bash[29936]: 4a6fe0b16d63aae1feb9a083bd969990090f8f1ccf0a30d99bb56ceb5aefede1 2024-06-01T01:51:44.884 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:44 smithi116 systemd[1]: Started Ceph mon.c for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T01:51:49.797 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:51:49] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:51:49.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T01:51:49.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[25082]: mon.a calling monitor election 2024-06-01T01:51:49.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:51:49.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[25082]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:51:49.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:51:49.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:51:49.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[25082]: mon.c calling monitor election 2024-06-01T01:51:49.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[25082]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:51:49.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:51:49.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:51:49.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:51:49.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[25082]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-01T01:51:49.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[25082]: monmap e2: 2 mons at {a=[v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0],c=[v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0]} 2024-06-01T01:51:49.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[25082]: fsmap 2024-06-01T01:51:49.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[25082]: osdmap e4: 0 total, 0 up, 0 in 2024-06-01T01:51:49.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[25082]: mgrmap e13: y(active, since 36s) 2024-06-01T01:51:49.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[25082]: overall HEALTH_OK 2024-06-01T01:51:49.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:49.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:49.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T01:51:49.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:51:49.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:49.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T01:51:49.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[29958]: mon.a calling monitor election 2024-06-01T01:51:49.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:51:49.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[29958]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:51:49.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:51:49.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:51:49.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[29958]: mon.c calling monitor election 2024-06-01T01:51:49.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[29958]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:51:49.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:51:49.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:51:49.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:51:49.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[29958]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-01T01:51:49.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[29958]: monmap e2: 2 mons at {a=[v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0],c=[v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0]} 2024-06-01T01:51:49.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[29958]: fsmap 2024-06-01T01:51:49.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[29958]: osdmap e4: 0 total, 0 up, 0 in 2024-06-01T01:51:49.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[29958]: mgrmap e13: y(active, since 36s) 2024-06-01T01:51:49.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[29958]: overall HEALTH_OK 2024-06-01T01:51:49.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:49.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:49.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T01:51:49.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:51:49.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:49 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:50.553 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-06-01T01:51:50.553 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":2,"fsid":"393f964a-1fb9-11ef-bc9b-c7b262605968","modified":"2024-06-01T01:51:44.526412Z","created":"2024-06-01T01:50:20.260187Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:3300","nonce":0},{"type":"v1","addr":"172.21.15.116:6789","nonce":0}]},"addr":"172.21.15.116:6789/0","public_addr":"172.21.15.116:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:3301","nonce":0},{"type":"v1","addr":"172.21.15.116:6790","nonce":0}]},"addr":"172.21.15.116:6790/0","public_addr":"172.21.15.116:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-01T01:51:50.555 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 2 2024-06-01T01:51:51.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:50 smithi116 ceph-mon[25082]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:51:51.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:50 smithi116 ceph-mon[25082]: Deploying daemon mon.b on smithi161 2024-06-01T01:51:51.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:50 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:51:51.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:50 smithi116 ceph-mon[25082]: from='client.? 172.21.15.161:0/507204349' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-01T01:51:51.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:50 smithi116 ceph-mon[29958]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:51:51.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:50 smithi116 ceph-mon[29958]: Deploying daemon mon.b on smithi161 2024-06-01T01:51:51.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:50 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:51:51.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:50 smithi116 ceph-mon[29958]: from='client.? 172.21.15.161:0/507204349' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-01T01:51:52.101 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-01T01:51:52.101 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-01T01:51:53.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:52 smithi116 ceph-mon[25082]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:51:53.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:52 smithi116 ceph-mon[29958]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:51:54.371 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-06-01T01:51:54.371 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":2,"fsid":"393f964a-1fb9-11ef-bc9b-c7b262605968","modified":"2024-06-01T01:51:44.526412Z","created":"2024-06-01T01:50:20.260187Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:3300","nonce":0},{"type":"v1","addr":"172.21.15.116:6789","nonce":0}]},"addr":"172.21.15.116:6789/0","public_addr":"172.21.15.116:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:3301","nonce":0},{"type":"v1","addr":"172.21.15.116:6790","nonce":0}]},"addr":"172.21.15.116:6790/0","public_addr":"172.21.15.116:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-01T01:51:54.372 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 2 2024-06-01T01:51:54.500 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-01T01:51:54.500 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-01T01:51:54.500 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-01T01:51:54.500 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1266072926' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-01T01:51:54.500 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: osdmap e4: 0 total, 0 up, 0 in 2024-06-01T01:51:54.500 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:51:54.501 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi116:172.21.15.116=a;smithi116:[v2:172.21.15.116:3301,v1:172.21.15.116:6790]=c;smithi161:172.21.15.161=b", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:51:54.501 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: Saving service mon spec with placement smithi116:172.21.15.116=a;smithi116:[v2:172.21.15.116:3301,v1:172.21.15.116:6790]=c;smithi161:172.21.15.161=b;count:3 2024-06-01T01:51:54.501 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:54.501 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:51:54.501 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:51:54.501 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:51:54.501 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:54.501 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T01:51:54.501 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:51:54.501 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: Deploying daemon mon.c on smithi116 2024-06-01T01:51:54.501 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:51:54.501 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: from='client.? 172.21.15.161:0/4186891177' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-01T01:51:54.501 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:51:54.501 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T01:51:54.502 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: mon.a calling monitor election 2024-06-01T01:51:54.502 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:51:54.502 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:51:54.502 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:51:54.502 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:51:54.502 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: mon.c calling monitor election 2024-06-01T01:51:54.502 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:51:54.502 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:51:54.502 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:51:54.502 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:51:54.502 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-01T01:51:54.502 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: monmap e2: 2 mons at {a=[v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0],c=[v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0]} 2024-06-01T01:51:54.502 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: fsmap 2024-06-01T01:51:54.503 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: osdmap e4: 0 total, 0 up, 0 in 2024-06-01T01:51:54.503 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: mgrmap e13: y(active, since 36s) 2024-06-01T01:51:54.503 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: overall HEALTH_OK 2024-06-01T01:51:54.503 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:54.503 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:54.503 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T01:51:54.503 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:51:54.503 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:54.503 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:51:54.503 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: Deploying daemon mon.b on smithi161 2024-06-01T01:51:54.503 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:51:54.503 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: from='client.? 172.21.15.161:0/507204349' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-01T01:51:54.503 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:51:54.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: mon.b@-1(synchronizing).paxosservice(auth 1..5) refresh upgraded, format 0 -> 3 2024-06-01T01:51:54.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 ceph-mon[26055]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete,mon_cluster_log_to_file 2024-06-01T01:51:54.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 bash[26024]: 9011986aa6e3b94168856b319ec79eb6ca37fd50c21f6d813d4b4ada0047ac78 2024-06-01T01:51:54.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:51:54 smithi161 systemd[1]: Started Ceph mon.b for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T01:51:56.068 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-01T01:51:56.069 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-01T01:51:59.788 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:51:59] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:51:59.804 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:51:59] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:51:59.805 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T01:51:59.805 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[29958]: mon.c calling monitor election 2024-06-01T01:51:59.805 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[29958]: mon.a calling monitor election 2024-06-01T01:51:59.805 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:51:59.805 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:51:59.805 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[29958]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:51:59.805 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:51:59.805 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:51:59.805 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[29958]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:51:59.805 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:51:59.806 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:51:59.806 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:51:59.806 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[29958]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-01T01:51:59.806 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[29958]: monmap e3: 3 mons at {a=[v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0],b=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0],c=[v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0]} 2024-06-01T01:51:59.806 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[29958]: fsmap 2024-06-01T01:51:59.806 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[29958]: osdmap e4: 0 total, 0 up, 0 in 2024-06-01T01:51:59.806 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[29958]: mgrmap e13: y(active, since 46s) 2024-06-01T01:51:59.806 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[29958]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-06-01T01:51:59.806 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[29958]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-06-01T01:51:59.806 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[29958]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-06-01T01:51:59.806 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[29958]: mon.b (rank 2) addr [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] is down (out of quorum) 2024-06-01T01:51:59.806 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:59.806 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:59.806 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:51:59.807 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:51:59.807 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:59.807 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:51:59.807 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T01:51:59.807 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[25082]: mon.c calling monitor election 2024-06-01T01:51:59.807 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[25082]: mon.a calling monitor election 2024-06-01T01:51:59.807 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:51:59.807 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:51:59.807 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[25082]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:51:59.807 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:51:59.808 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:51:59.808 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[25082]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:51:59.808 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:51:59.808 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:51:59.808 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:51:59.808 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[25082]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-01T01:51:59.808 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[25082]: monmap e3: 3 mons at {a=[v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0],b=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0],c=[v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0]} 2024-06-01T01:51:59.808 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[25082]: fsmap 2024-06-01T01:51:59.808 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[25082]: osdmap e4: 0 total, 0 up, 0 in 2024-06-01T01:51:59.808 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[25082]: mgrmap e13: y(active, since 46s) 2024-06-01T01:51:59.808 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[25082]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-06-01T01:51:59.808 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[25082]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-06-01T01:51:59.808 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[25082]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-06-01T01:51:59.808 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[25082]: mon.b (rank 2) addr [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] is down (out of quorum) 2024-06-01T01:51:59.809 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:59.809 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:59.809 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:51:59.809 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:51:59.809 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:51:59.809 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:51:59 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:51:59.995 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-06-01T01:51:59.995 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":3,"fsid":"393f964a-1fb9-11ef-bc9b-c7b262605968","modified":"2024-06-01T01:51:54.507771Z","created":"2024-06-01T01:50:20.260187Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:3300","nonce":0},{"type":"v1","addr":"172.21.15.116:6789","nonce":0}]},"addr":"172.21.15.116:6789/0","public_addr":"172.21.15.116:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:3301","nonce":0},{"type":"v1","addr":"172.21.15.116:6790","nonce":0}]},"addr":"172.21.15.116:6790/0","public_addr":"172.21.15.116:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:3300","nonce":0},{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-01T01:51:59.997 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 3 2024-06-01T01:52:00.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:00 smithi116 ceph-mon[25082]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:00.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:00 smithi116 ceph-mon[25082]: Updating smithi116:/etc/ceph/ceph.conf 2024-06-01T01:52:00.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:00 smithi116 ceph-mon[25082]: Updating smithi161:/etc/ceph/ceph.conf 2024-06-01T01:52:00.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:00 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:00.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:00 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:00.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:00 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:00.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:00 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:00.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:00 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:00.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:00 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:52:00.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:00 smithi116 ceph-mon[25082]: Reconfiguring mon.a (unknown last config time)... 2024-06-01T01:52:00.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:00 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T01:52:00.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:00 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T01:52:00.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:00 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:00.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:00 smithi116 ceph-mon[25082]: Reconfiguring daemon mon.a on smithi116 2024-06-01T01:52:00.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:00 smithi116 ceph-mon[25082]: from='client.? 172.21.15.161:0/1895947201' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-01T01:52:00.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:00 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:52:00.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:00 smithi116 ceph-mon[29958]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:00.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:00 smithi116 ceph-mon[29958]: Updating smithi116:/etc/ceph/ceph.conf 2024-06-01T01:52:00.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:00 smithi116 ceph-mon[29958]: Updating smithi161:/etc/ceph/ceph.conf 2024-06-01T01:52:00.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:00 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:00.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:00 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:00.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:00 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:00.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:00 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:00.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:00 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:00.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:00 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:52:00.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:00 smithi116 ceph-mon[29958]: Reconfiguring mon.a (unknown last config time)... 2024-06-01T01:52:00.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:00 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T01:52:00.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:00 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T01:52:00.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:00 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:00.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:00 smithi116 ceph-mon[29958]: Reconfiguring daemon mon.a on smithi116 2024-06-01T01:52:00.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:00 smithi116 ceph-mon[29958]: from='client.? 172.21.15.161:0/1895947201' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-01T01:52:00.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:00 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:52:01.167 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-06-01T01:52:01.167 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph config generate-minimal-conf 2024-06-01T01:52:01.546 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:52:01 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:52:01] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:52:01.764 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T01:52:01.764 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: mon.c calling monitor election 2024-06-01T01:52:01.764 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: mon.a calling monitor election 2024-06-01T01:52:01.764 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:52:01.764 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:52:01.764 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:01.764 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:52:01.764 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:52:01.764 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:01.764 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:52:01.765 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:52:01.765 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:52:01.765 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-01T01:52:01.765 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: monmap e3: 3 mons at {a=[v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0],b=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0],c=[v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0]} 2024-06-01T01:52:01.765 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: fsmap 2024-06-01T01:52:01.765 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: osdmap e4: 0 total, 0 up, 0 in 2024-06-01T01:52:01.765 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: mgrmap e13: y(active, since 46s) 2024-06-01T01:52:01.765 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-06-01T01:52:01.765 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-06-01T01:52:01.765 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-06-01T01:52:01.765 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: mon.b (rank 2) addr [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] is down (out of quorum) 2024-06-01T01:52:01.765 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:01.765 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:01.765 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:52:01.765 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:01.766 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:01.766 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:52:01.766 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:01.766 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: Updating smithi116:/etc/ceph/ceph.conf 2024-06-01T01:52:01.766 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: Updating smithi161:/etc/ceph/ceph.conf 2024-06-01T01:52:01.766 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:01.766 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:01.766 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:01.766 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:01.766 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:01.766 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:52:01.766 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: Reconfiguring mon.a (unknown last config time)... 2024-06-01T01:52:01.766 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T01:52:01.767 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T01:52:01.767 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:01.767 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: Reconfiguring daemon mon.a on smithi116 2024-06-01T01:52:01.767 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: from='client.? 172.21.15.161:0/1895947201' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-01T01:52:01.767 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:01 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:52:02.046 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:52:01 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:52:01] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:52:02.395 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[29958]: mon.b calling monitor election 2024-06-01T01:52:02.395 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[29958]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:02.395 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[29958]: mon.b calling monitor election 2024-06-01T01:52:02.395 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[29958]: mon.c calling monitor election 2024-06-01T01:52:02.395 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[29958]: mon.a calling monitor election 2024-06-01T01:52:02.395 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[29958]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-01T01:52:02.395 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[29958]: monmap e3: 3 mons at {a=[v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0],b=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0],c=[v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0]} 2024-06-01T01:52:02.395 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[29958]: fsmap 2024-06-01T01:52:02.395 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[29958]: osdmap e4: 0 total, 0 up, 0 in 2024-06-01T01:52:02.395 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[29958]: mgrmap e13: y(active, since 48s) 2024-06-01T01:52:02.396 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[29958]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-06-01T01:52:02.396 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[29958]: Cluster is now healthy 2024-06-01T01:52:02.396 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[29958]: overall HEALTH_OK 2024-06-01T01:52:02.396 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:02.396 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[29958]: Reconfiguring mon.c (monmap changed)... 2024-06-01T01:52:02.396 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T01:52:02.396 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T01:52:02.396 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:02.396 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[29958]: Reconfiguring daemon mon.c on smithi116 2024-06-01T01:52:02.396 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:02.396 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:02.397 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[25082]: mon.b calling monitor election 2024-06-01T01:52:02.397 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[25082]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:02.397 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[25082]: mon.b calling monitor election 2024-06-01T01:52:02.397 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[25082]: mon.c calling monitor election 2024-06-01T01:52:02.397 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[25082]: mon.a calling monitor election 2024-06-01T01:52:02.397 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[25082]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-01T01:52:02.397 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[25082]: monmap e3: 3 mons at {a=[v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0],b=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0],c=[v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0]} 2024-06-01T01:52:02.397 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[25082]: fsmap 2024-06-01T01:52:02.397 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[25082]: osdmap e4: 0 total, 0 up, 0 in 2024-06-01T01:52:02.397 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[25082]: mgrmap e13: y(active, since 48s) 2024-06-01T01:52:02.397 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[25082]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-06-01T01:52:02.398 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[25082]: Cluster is now healthy 2024-06-01T01:52:02.398 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[25082]: overall HEALTH_OK 2024-06-01T01:52:02.398 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:02.398 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[25082]: Reconfiguring mon.c (monmap changed)... 2024-06-01T01:52:02.398 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T01:52:02.398 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T01:52:02.398 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:02.398 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[25082]: Reconfiguring daemon mon.c on smithi116 2024-06-01T01:52:02.398 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:02.398 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:02 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:02.548 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:02 smithi161 ceph-mon[26055]: mon.b calling monitor election 2024-06-01T01:52:02.548 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:02 smithi161 ceph-mon[26055]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:02.548 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:02 smithi161 ceph-mon[26055]: mon.b calling monitor election 2024-06-01T01:52:02.548 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:02 smithi161 ceph-mon[26055]: mon.c calling monitor election 2024-06-01T01:52:02.548 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:02 smithi161 ceph-mon[26055]: mon.a calling monitor election 2024-06-01T01:52:02.548 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:02 smithi161 ceph-mon[26055]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-01T01:52:02.548 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:02 smithi161 ceph-mon[26055]: monmap e3: 3 mons at {a=[v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0],b=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0],c=[v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0]} 2024-06-01T01:52:02.548 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:02 smithi161 ceph-mon[26055]: fsmap 2024-06-01T01:52:02.548 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:02 smithi161 ceph-mon[26055]: osdmap e4: 0 total, 0 up, 0 in 2024-06-01T01:52:02.548 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:02 smithi161 ceph-mon[26055]: mgrmap e13: y(active, since 48s) 2024-06-01T01:52:02.548 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:02 smithi161 ceph-mon[26055]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-06-01T01:52:02.549 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:02 smithi161 ceph-mon[26055]: Cluster is now healthy 2024-06-01T01:52:02.549 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:02 smithi161 ceph-mon[26055]: overall HEALTH_OK 2024-06-01T01:52:02.549 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:02 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:02.549 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:02 smithi161 ceph-mon[26055]: Reconfiguring mon.c (monmap changed)... 2024-06-01T01:52:02.549 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:02 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T01:52:02.549 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:02 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T01:52:02.549 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:02 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:02.549 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:02 smithi161 ceph-mon[26055]: Reconfiguring daemon mon.c on smithi116 2024-06-01T01:52:02.549 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:02 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:02.549 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:02 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:03.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:03 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:52:03.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:03 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:03.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:03 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:03.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:03 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:03.546 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:52:03 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:52:03] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:52:03.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:03 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:52:03.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:03 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:03.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:03 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:03.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:03 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:03.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:03 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:52:03.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:03 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:03.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:03 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:03.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:03 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:04.474 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:04 smithi116 ceph-mon[29958]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:04.474 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:04 smithi116 ceph-mon[25082]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:04.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:04 smithi161 ceph-mon[26055]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:04.787 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:52:04 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:52:04] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:52:05.599 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:05 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:05.599 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:05 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:05.600 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:05 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:05.600 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:05 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:05.619 INFO:teuthology.orchestra.run.smithi116.stdout:# minimal ceph.conf for 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T01:52:05.619 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2024-06-01T01:52:05.619 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T01:52:05.620 INFO:teuthology.orchestra.run.smithi116.stdout: mon_host = [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] 2024-06-01T01:52:05.922 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:05 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:05.922 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:05 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:06.493 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-06-01T01:52:06.493 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-06-01T01:52:06.493 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/ceph/ceph.conf 2024-06-01T01:52:06.522 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-06-01T01:52:06.522 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-01T01:52:06.588 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-06-01T01:52:06.588 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.conf 2024-06-01T01:52:06.615 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-06-01T01:52:06.615 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-01T01:52:06.680 INFO:tasks.cephadm:Adding mgr.y on smithi116 2024-06-01T01:52:06.680 INFO:tasks.cephadm:Adding mgr.x on smithi161 2024-06-01T01:52:06.680 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph orch apply mgr '2;smithi116=y;smithi161=x' 2024-06-01T01:52:06.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:06 smithi116 ceph-mon[25082]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:06.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:06 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/2326588720' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:06.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:06 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:06.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:06 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T01:52:06.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:06 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T01:52:06.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:06 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:06.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:06 smithi116 ceph-mon[29958]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:06.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:06 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/2326588720' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:06.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:06 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:06.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:06 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T01:52:06.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:06 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T01:52:06.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:06 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:06.888 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:06 smithi161 ceph-mon[26055]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:06.888 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:06 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/2326588720' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:06.888 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:06 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:06.888 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:06 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T01:52:06.888 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:06 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T01:52:06.888 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:06 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:07.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:07 smithi161 ceph-mon[26055]: Reconfiguring mon.b (monmap changed)... 2024-06-01T01:52:07.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:07 smithi161 ceph-mon[26055]: Reconfiguring daemon mon.b on smithi161 2024-06-01T01:52:07.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:07 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:07.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:07 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:52:07.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:07 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:07.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:07 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:52:07.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:07 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:08.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:07 smithi116 ceph-mon[25082]: Reconfiguring mon.b (monmap changed)... 2024-06-01T01:52:08.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:07 smithi116 ceph-mon[25082]: Reconfiguring daemon mon.b on smithi161 2024-06-01T01:52:08.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:07 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:08.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:07 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:52:08.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:07 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:08.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:07 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:52:08.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:07 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:08.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:07 smithi116 ceph-mon[29958]: Reconfiguring mon.b (monmap changed)... 2024-06-01T01:52:08.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:07 smithi116 ceph-mon[29958]: Reconfiguring daemon mon.b on smithi161 2024-06-01T01:52:08.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:07 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:08.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:07 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:52:08.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:07 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:08.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:07 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:52:08.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:07 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:08.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:08 smithi161 ceph-mon[26055]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:08.812 INFO:teuthology.orchestra.run.smithi161.stdout:Scheduled mgr update... 2024-06-01T01:52:09.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:08 smithi116 ceph-mon[25082]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:09.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:08 smithi116 ceph-mon[29958]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:09.400 DEBUG:teuthology.orchestra.run.smithi161:mgr.x> sudo journalctl -f -n 0 -u ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mgr.x.service 2024-06-01T01:52:09.402 INFO:tasks.cephadm:Deploying OSDs... 2024-06-01T01:52:09.403 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-06-01T01:52:09.403 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/scratch_devs of=/dev/stdout 2024-06-01T01:52:09.423 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-01T01:52:09.423 DEBUG:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_1 2024-06-01T01:52:09.481 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-01T01:52:09.481 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-01T01:52:09.482 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-06-01T01:52:09.482 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-01T01:52:09.482 INFO:teuthology.orchestra.run.smithi116.stdout:Context: system_u:object_r:device_t:s0 2024-06-01T01:52:09.482 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2024-06-01 01:52:03.727353790 +0000 2024-06-01T01:52:09.482 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2024-06-01 01:48:46.283972354 +0000 2024-06-01T01:52:09.482 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2024-06-01 01:48:46.283972354 +0000 2024-06-01T01:52:09.482 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: 2024-06-01 01:48:46.283972354 +0000 2024-06-01T01:52:09.482 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-01T01:52:09.549 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2024-06-01T01:52:09.549 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2024-06-01T01:52:09.549 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.000225733 s, 2.3 MB/s 2024-06-01T01:52:09.551 DEBUG:teuthology.orchestra.run.smithi116:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-01T01:52:09.608 DEBUG:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_2 2024-06-01T01:52:09.665 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-01T01:52:09.665 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-01T01:52:09.665 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-06-01T01:52:09.666 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-01T01:52:09.666 INFO:teuthology.orchestra.run.smithi116.stdout:Context: system_u:object_r:device_t:s0 2024-06-01T01:52:09.666 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2024-06-01 01:52:03.727353790 +0000 2024-06-01T01:52:09.666 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2024-06-01 01:48:46.624966655 +0000 2024-06-01T01:52:09.666 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2024-06-01 01:48:46.624966655 +0000 2024-06-01T01:52:09.666 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: 2024-06-01 01:48:46.624966655 +0000 2024-06-01T01:52:09.666 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-01T01:52:09.731 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2024-06-01T01:52:09.731 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2024-06-01T01:52:09.731 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.000227754 s, 2.2 MB/s 2024-06-01T01:52:09.732 DEBUG:teuthology.orchestra.run.smithi116:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-01T01:52:09.789 DEBUG:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_3 2024-06-01T01:52:09.846 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-01T01:52:09.846 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-01T01:52:09.846 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-06-01T01:52:09.846 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-01T01:52:09.847 INFO:teuthology.orchestra.run.smithi116.stdout:Context: system_u:object_r:device_t:s0 2024-06-01T01:52:09.847 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2024-06-01 01:52:03.727353790 +0000 2024-06-01T01:52:09.847 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2024-06-01 01:48:46.951961191 +0000 2024-06-01T01:52:09.847 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2024-06-01 01:48:46.951961191 +0000 2024-06-01T01:52:09.847 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: 2024-06-01 01:48:46.951961191 +0000 2024-06-01T01:52:09.847 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-01T01:52:09.911 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2024-06-01T01:52:09.912 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2024-06-01T01:52:09.912 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.000190934 s, 2.7 MB/s 2024-06-01T01:52:09.913 DEBUG:teuthology.orchestra.run.smithi116:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-01T01:52:09.914 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:09 smithi161 ceph-mon[26055]: from='client.24107 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi116=y;smithi161=x", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:52:09.914 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:09 smithi161 ceph-mon[26055]: Saving service mgr spec with placement smithi116=y;smithi161=x;count:2 2024-06-01T01:52:09.914 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:09 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:09.915 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:09 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:52:09.915 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:09 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:09.915 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:09 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:52:09.915 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:09 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:09.915 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:09 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T01:52:09.915 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:09 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-01T01:52:09.915 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:09 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-01T01:52:09.915 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:09 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:09.915 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:09 smithi161 ceph-mon[26055]: Deploying daemon mgr.x on smithi161 2024-06-01T01:52:09.969 DEBUG:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_4 2024-06-01T01:52:10.025 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-01T01:52:10.026 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-01T01:52:10.026 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-06-01T01:52:10.026 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-01T01:52:10.026 INFO:teuthology.orchestra.run.smithi116.stdout:Context: system_u:object_r:device_t:s0 2024-06-01T01:52:10.026 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2024-06-01 01:52:03.727353790 +0000 2024-06-01T01:52:10.026 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2024-06-01 01:48:47.290955525 +0000 2024-06-01T01:52:10.026 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2024-06-01 01:48:47.290955525 +0000 2024-06-01T01:52:10.026 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: 2024-06-01 01:48:47.290955525 +0000 2024-06-01T01:52:10.027 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-01T01:52:10.087 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:09 smithi116 ceph-mon[29958]: from='client.24107 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi116=y;smithi161=x", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:52:10.087 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:09 smithi116 ceph-mon[29958]: Saving service mgr spec with placement smithi116=y;smithi161=x;count:2 2024-06-01T01:52:10.087 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:09 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:10.087 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:09 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:52:10.087 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:09 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:10.087 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:09 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:52:10.087 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:09 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:10.087 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:09 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T01:52:10.087 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:09 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-01T01:52:10.087 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:09 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-01T01:52:10.088 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:09 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:10.088 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:09 smithi116 ceph-mon[29958]: Deploying daemon mgr.x on smithi161 2024-06-01T01:52:10.088 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:09 smithi116 ceph-mon[25082]: from='client.24107 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi116=y;smithi161=x", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:52:10.088 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:09 smithi116 ceph-mon[25082]: Saving service mgr spec with placement smithi116=y;smithi161=x;count:2 2024-06-01T01:52:10.088 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:09 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:10.088 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:09 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:52:10.088 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:09 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:10.088 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:09 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:52:10.088 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:09 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:10.088 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:09 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T01:52:10.089 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:09 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-01T01:52:10.089 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:09 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-01T01:52:10.089 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:09 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:10.089 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:09 smithi116 ceph-mon[25082]: Deploying daemon mgr.x on smithi161 2024-06-01T01:52:10.089 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2024-06-01T01:52:10.089 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2024-06-01T01:52:10.089 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.000206188 s, 2.5 MB/s 2024-06-01T01:52:10.090 DEBUG:teuthology.orchestra.run.smithi116:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-01T01:52:10.147 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-06-01T01:52:10.147 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/scratch_devs of=/dev/stdout 2024-06-01T01:52:10.163 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-01T01:52:10.163 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_1 2024-06-01T01:52:10.218 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-01T01:52:10.218 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-01T01:52:10.218 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-06-01T01:52:10.218 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-01T01:52:10.218 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-06-01T01:52:10.218 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-06-01 01:52:02.490043915 +0000 2024-06-01T01:52:10.219 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-06-01 01:48:50.920727584 +0000 2024-06-01T01:52:10.219 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-06-01 01:48:50.920727584 +0000 2024-06-01T01:52:10.219 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-06-01 01:48:50.920727584 +0000 2024-06-01T01:52:10.219 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-01T01:52:10.282 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-06-01T01:52:10.282 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-06-01T01:52:10.282 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000268805 s, 1.9 MB/s 2024-06-01T01:52:10.284 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-01T01:52:10.341 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_2 2024-06-01T01:52:10.397 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-01T01:52:10.397 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-01T01:52:10.397 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-06-01T01:52:10.397 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-01T01:52:10.397 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-06-01T01:52:10.398 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-06-01 01:52:02.490043915 +0000 2024-06-01T01:52:10.398 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-06-01 01:48:51.253719960 +0000 2024-06-01T01:52:10.398 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-06-01 01:48:51.253719960 +0000 2024-06-01T01:52:10.398 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-06-01 01:48:51.253719960 +0000 2024-06-01T01:52:10.399 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-01T01:52:10.460 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-06-01T01:52:10.460 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-06-01T01:52:10.460 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000197022 s, 2.6 MB/s 2024-06-01T01:52:10.461 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-01T01:52:10.519 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_3 2024-06-01T01:52:10.575 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-01T01:52:10.575 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-01T01:52:10.576 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-06-01T01:52:10.576 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-01T01:52:10.576 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-06-01T01:52:10.576 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-06-01 01:52:02.490043915 +0000 2024-06-01T01:52:10.576 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-06-01 01:48:51.592712199 +0000 2024-06-01T01:52:10.576 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-06-01 01:48:51.592712199 +0000 2024-06-01T01:52:10.576 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-06-01 01:48:51.592712199 +0000 2024-06-01T01:52:10.576 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-01T01:52:10.647 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-06-01T01:52:10.647 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-06-01T01:52:10.648 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000246182 s, 2.1 MB/s 2024-06-01T01:52:10.649 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-01T01:52:10.705 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_4 2024-06-01T01:52:10.760 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-01T01:52:10.760 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-01T01:52:10.761 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-06-01T01:52:10.761 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-01T01:52:10.761 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-06-01T01:52:10.761 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-06-01 01:52:10.606863996 +0000 2024-06-01T01:52:10.761 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-06-01 01:48:51.930704461 +0000 2024-06-01T01:52:10.761 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-06-01 01:48:51.930704461 +0000 2024-06-01T01:52:10.761 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-06-01 01:48:51.930704461 +0000 2024-06-01T01:52:10.761 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-01T01:52:10.826 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-06-01T01:52:10.826 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-06-01T01:52:10.826 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000208849 s, 2.5 MB/s 2024-06-01T01:52:10.827 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-01T01:52:10.884 INFO:tasks.cephadm:Deploying osd.0 on smithi116 with /dev/vg_nvme/lv_4... 2024-06-01T01:52:10.884 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-01T01:52:11.130 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:10 smithi161 ceph-mon[26055]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:11.197 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:10 smithi116 ceph-mon[25082]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:11.197 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:10 smithi116 ceph-mon[29958]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:11.437 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:52:11 smithi161 ceph-mgr[27604]: -- 172.21.15.161:0/3751614905 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560e752c6340 con 0x560e7529e800 2024-06-01T01:52:11.697 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:52:11 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:52:11.437+0000 7fde76c10000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-01T01:52:11.697 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:52:11 smithi161 bash[27584]: f31540b9633899be6dfdb70ccfb8d6d8daa0e75b02e630a4f31f3d5690a380e3 2024-06-01T01:52:11.697 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:52:11 smithi161 systemd[1]: Started Ceph mgr.x for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T01:52:11.697 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:52:11 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:52:11.571+0000 7fde76c10000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-01T01:52:11.697 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:52:11 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:52:11.670+0000 7fde76c10000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-01T01:52:12.447 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:52:11 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:52:11.976+0000 7fde76c10000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-01T01:52:12.447 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:52:12 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:52:12.058+0000 7fde76c10000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-01T01:52:12.773 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:12 smithi116 ceph-mon[25082]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:12.774 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:12 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:12.774 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:12 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:52:12.774 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:12 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:12.774 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:12 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:52:12.774 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:12 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:12.774 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:12 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:52:12.774 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:12 smithi116 ceph-mon[25082]: Reconfiguring mgr.y (unknown last config time)... 2024-06-01T01:52:12.774 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:12 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T01:52:12.774 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:12 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-01T01:52:12.774 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:12 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:12.774 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:12 smithi116 ceph-mon[25082]: Reconfiguring daemon mgr.y on smithi116 2024-06-01T01:52:12.775 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:12 smithi116 ceph-mon[29958]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:12.775 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:12 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:12.775 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:12 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:52:12.775 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:12 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:12.775 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:12 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:52:12.775 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:12 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:12.775 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:12 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:52:12.775 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:12 smithi116 ceph-mon[29958]: Reconfiguring mgr.y (unknown last config time)... 2024-06-01T01:52:12.775 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:12 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T01:52:12.775 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:12 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-01T01:52:12.775 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:12 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:12.775 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:12 smithi116 ceph-mon[29958]: Reconfiguring daemon mgr.y on smithi116 2024-06-01T01:52:12.781 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:12 smithi161 ceph-mon[26055]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:12.781 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:12 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:12.781 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:12 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:52:12.781 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:12 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:12.781 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:12 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:52:12.781 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:12 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:12.781 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:12 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:52:12.782 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:12 smithi161 ceph-mon[26055]: Reconfiguring mgr.y (unknown last config time)... 2024-06-01T01:52:12.782 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:12 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T01:52:12.782 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:12 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-01T01:52:12.782 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:12 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:12.782 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:12 smithi161 ceph-mon[26055]: Reconfiguring daemon mgr.y on smithi116 2024-06-01T01:52:13.351 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:52:13 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:52:13.233+0000 7fde76c10000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-01T01:52:13.603 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:52:13 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:52:13.405+0000 7fde76c10000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-01T01:52:13.603 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:52:13 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:52:13.487+0000 7fde76c10000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-01T01:52:13.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:13 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T01:52:13.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:13 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T01:52:13.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:13 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:13.703 INFO:teuthology.orchestra.run.smithi116.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-01T01:52:13.704 INFO:teuthology.orchestra.run.smithi116.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-01T01:52:13.704 INFO:teuthology.orchestra.run.smithi116.stderr: stderr: 10+0 records in 2024-06-01T01:52:13.704 INFO:teuthology.orchestra.run.smithi116.stderr:10+0 records out 2024-06-01T01:52:13.704 INFO:teuthology.orchestra.run.smithi116.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112909 s, 929 MB/s 2024-06-01T01:52:13.704 INFO:teuthology.orchestra.run.smithi116.stderr:--> Zapping successful for: 2024-06-01T01:52:14.023 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:52:13 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:52:13.808+0000 7fde76c10000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-01T01:52:14.024 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:52:13 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:52:13.911+0000 7fde76c10000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-01T01:52:14.024 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:52:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:52:14.024+0000 7fde76c10000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-01T01:52:14.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:13 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T01:52:14.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:13 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T01:52:14.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:13 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:14.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:13 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T01:52:14.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:13 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T01:52:14.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:13 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:14.222 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi116:vg_nvme/lv_4 2024-06-01T01:52:14.654 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:52:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:52:14.289+0000 7fde76c10000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-01T01:52:14.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:14 smithi116 ceph-mon[29958]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:14.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:14 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:14.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:14 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:52:14.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:14 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:14.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:14 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:52:14.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:14 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:14.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:14 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:52:14.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:14 smithi116 ceph-mon[25082]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:14.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:14 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:14.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:14 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:52:14.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:14 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:14.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:14 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:52:14.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:14 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:14.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:14 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:52:14.946 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:52:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:52:14.756+0000 7fde76c10000 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-01T01:52:14.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:14 smithi161 ceph-mon[26055]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:14.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:14 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:14.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:14 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:52:14.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:14 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:14.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:14 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:52:14.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:14 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:14.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:14 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:52:15.341 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:52:15 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:52:15.249+0000 7fde76c10000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-01T01:52:15.510 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:52:15 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:52:15] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:52:15.644 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:52:15 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:52:15.342+0000 7fde76c10000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-01T01:52:15.644 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:52:15 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:52:15.644+0000 7fde76c10000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-01T01:52:15.791 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:52:15 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:52:15] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:52:15.947 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:52:15 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:52:15.732+0000 7fde76c10000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-01T01:52:16.447 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:52:16 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:52:16.137+0000 7fde76c10000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-01T01:52:16.447 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:52:16 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:52:16.269+0000 7fde76c10000 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-01T01:52:16.447 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:52:16 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:52:16.360+0000 7fde76c10000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-01T01:52:16.745 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:16 smithi116 ceph-mon[25082]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:16.745 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:16 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:16.745 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:16 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:16.745 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:16 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:16.746 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:16 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:52:16.746 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:16 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:16.746 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:16 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:52:16.746 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:16 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:16.746 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:16 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-01T01:52:16.746 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:16 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-01T01:52:16.746 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:16 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:16.746 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:16 smithi116 ceph-mon[29958]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:16.746 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:16 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:16.746 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:16 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:16.746 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:16 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:16.747 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:16 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:52:16.747 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:16 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:16.747 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:16 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:52:16.747 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:16 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:16.747 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:16 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-01T01:52:16.747 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:16 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-01T01:52:16.747 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:16 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:16.947 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:52:16 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:52:16.473+0000 7fde76c10000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-01T01:52:16.947 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:52:16 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:52:16.561+0000 7fde76c10000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-01T01:52:16.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:16 smithi161 ceph-mon[26055]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:16.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:16 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:16.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:16 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:16.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:16 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:16.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:16 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:52:16.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:16 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:16.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:16 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:52:16.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:16 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:16.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:16 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-01T01:52:16.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:16 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-01T01:52:16.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:16 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:17.446 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:52:17 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:52:17.100+0000 7fde76c10000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-01T01:52:17.722 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:17 smithi116 ceph-mon[25082]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:52:17.722 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:17 smithi116 ceph-mon[25082]: Standby manager daemon x started 2024-06-01T01:52:17.722 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:17 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.161:0/1621300166' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-01T01:52:17.722 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:17 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.161:0/1621300166' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T01:52:17.722 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:17 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.161:0/1621300166' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-01T01:52:17.722 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:17 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.161:0/1621300166' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T01:52:17.722 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:17 smithi116 ceph-mon[29958]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:52:17.723 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:17 smithi116 ceph-mon[29958]: Standby manager daemon x started 2024-06-01T01:52:17.723 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:17 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.161:0/1621300166' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-01T01:52:17.723 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:17 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.161:0/1621300166' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T01:52:17.723 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:17 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.161:0/1621300166' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-01T01:52:17.723 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:17 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.161:0/1621300166' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T01:52:17.947 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:52:17 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:52:17.461+0000 7fde76c10000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-01T01:52:17.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:17 smithi161 ceph-mon[26055]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:52:17.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:17 smithi161 ceph-mon[26055]: Standby manager daemon x started 2024-06-01T01:52:17.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:17 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.161:0/1621300166' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-01T01:52:17.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:17 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.161:0/1621300166' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T01:52:17.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:17 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.161:0/1621300166' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-01T01:52:17.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:17 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.161:0/1621300166' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T01:52:18.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:18 smithi116 ceph-mon[25082]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:18.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:18 smithi116 ceph-mon[25082]: mgrmap e14: y(active, since 64s), standbys: x 2024-06-01T01:52:18.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:18 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-01T01:52:18.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:18 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/3964006949' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77d7478a-9bf5-473d-8296-b0c91060b912"}]: dispatch 2024-06-01T01:52:18.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:18 smithi116 ceph-mon[25082]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77d7478a-9bf5-473d-8296-b0c91060b912"}]: dispatch 2024-06-01T01:52:18.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:18 smithi116 ceph-mon[25082]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77d7478a-9bf5-473d-8296-b0c91060b912"}]': finished 2024-06-01T01:52:18.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:18 smithi116 ceph-mon[25082]: osdmap e5: 1 total, 0 up, 1 in 2024-06-01T01:52:18.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:18 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T01:52:18.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:18 smithi116 ceph-mon[29958]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:18.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:18 smithi116 ceph-mon[29958]: mgrmap e14: y(active, since 64s), standbys: x 2024-06-01T01:52:18.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:18 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-01T01:52:18.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:18 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/3964006949' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77d7478a-9bf5-473d-8296-b0c91060b912"}]: dispatch 2024-06-01T01:52:18.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:18 smithi116 ceph-mon[29958]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77d7478a-9bf5-473d-8296-b0c91060b912"}]: dispatch 2024-06-01T01:52:18.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:18 smithi116 ceph-mon[29958]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77d7478a-9bf5-473d-8296-b0c91060b912"}]': finished 2024-06-01T01:52:18.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:18 smithi116 ceph-mon[29958]: osdmap e5: 1 total, 0 up, 1 in 2024-06-01T01:52:18.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:18 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T01:52:18.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:18 smithi161 ceph-mon[26055]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:18.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:18 smithi161 ceph-mon[26055]: mgrmap e14: y(active, since 64s), standbys: x 2024-06-01T01:52:18.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:18 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-01T01:52:18.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:18 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/3964006949' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77d7478a-9bf5-473d-8296-b0c91060b912"}]: dispatch 2024-06-01T01:52:18.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:18 smithi161 ceph-mon[26055]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77d7478a-9bf5-473d-8296-b0c91060b912"}]: dispatch 2024-06-01T01:52:18.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:18 smithi161 ceph-mon[26055]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77d7478a-9bf5-473d-8296-b0c91060b912"}]': finished 2024-06-01T01:52:18.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:18 smithi161 ceph-mon[26055]: osdmap e5: 1 total, 0 up, 1 in 2024-06-01T01:52:18.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:18 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T01:52:20.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:20 smithi116 ceph-mon[25082]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:20.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:20 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1491706604' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-01T01:52:20.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:20 smithi116 ceph-mon[29958]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:20.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:20 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1491706604' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-01T01:52:20.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:20 smithi161 ceph-mon[26055]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:20.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:20 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1491706604' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-01T01:52:22.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:22 smithi116 ceph-mon[25082]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:22.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:22 smithi116 ceph-mon[29958]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:22 smithi161 ceph-mon[26055]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:23.795 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:52:23 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:52:23] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:52:24.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:24 smithi161 ceph-mon[26055]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:24.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:24 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:24.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:24 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:25.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:24 smithi116 ceph-mon[25082]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:25.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:24 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:25.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:24 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:25.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:24 smithi116 ceph-mon[29958]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:25.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:24 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:25.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:24 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:26.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:26 smithi116 ceph-mon[29958]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:26.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:26 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-01T01:52:26.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:26 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:26.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:26 smithi116 ceph-mon[29958]: Deploying daemon osd.0 on smithi116 2024-06-01T01:52:26.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:26 smithi116 ceph-mon[25082]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:26.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:26 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-01T01:52:26.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:26 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:26.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:26 smithi116 ceph-mon[25082]: Deploying daemon osd.0 on smithi116 2024-06-01T01:52:26.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:26 smithi161 ceph-mon[26055]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:26.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:26 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-01T01:52:26.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:26 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:26.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:26 smithi161 ceph-mon[26055]: Deploying daemon osd.0 on smithi116 2024-06-01T01:52:28.694 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:28 smithi116 ceph-mon[25082]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:28.694 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:28 smithi116 ceph-mon[29958]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:28.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:28 smithi161 ceph-mon[26055]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:30.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:30 smithi161 ceph-mon[26055]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:31.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:30 smithi116 ceph-mon[25082]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:31.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:30 smithi116 ceph-mon[29958]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:32.124 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:31 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:32.124 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:31 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:52:32.124 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:31 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:32.124 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:31 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:52:32.124 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:31 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:32.124 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:31 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:32.124 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:31 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:52:32.124 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:31 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:32.125 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:31 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:52:32.125 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:31 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:32.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:31 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:32.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:31 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:52:32.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:31 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:32.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:31 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:52:32.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:31 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:33.112 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:32 smithi161 ceph-mon[26055]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:33.213 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:32 smithi116 ceph-mon[25082]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:33.213 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:32 smithi116 ceph-mon[29958]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:33.519 INFO:teuthology.orchestra.run.smithi116.stdout:Created osd(s) 0 on host 'smithi116' 2024-06-01T01:52:33.795 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:52:33 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:52:33] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:52:34.223 DEBUG:teuthology.orchestra.run.smithi116:osd.0> sudo journalctl -f -n 0 -u ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.0.service 2024-06-01T01:52:34.226 INFO:tasks.cephadm:Deploying osd.1 on smithi116 with /dev/vg_nvme/lv_3... 2024-06-01T01:52:34.226 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-01T01:52:34.617 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:34 smithi161 ceph-mon[26055]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:34.617 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:34 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:34.617 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:34 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:52:34.618 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:34 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:34.618 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:34 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:52:34.618 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:34 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:52:34.618 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:34 smithi161 ceph-mon[26055]: Adjusting osd_memory_target on smithi116 to 15040M 2024-06-01T01:52:34.618 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:34 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:34.618 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:34 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:34.618 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:34 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:34.618 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:34 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:34.618 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:34 smithi161 ceph-mon[26055]: from='osd.0 [v2:172.21.15.116:6802/1460215654,v1:172.21.15.116:6803/1460215654]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-01T01:52:34.618 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:34 smithi161 ceph-mon[26055]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-01T01:52:34.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:34 smithi116 ceph-mon[25082]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:34.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:34 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:34.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:34 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:52:34.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:34 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:34.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:34 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:52:34.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:34 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:52:34.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:34 smithi116 ceph-mon[25082]: Adjusting osd_memory_target on smithi116 to 15040M 2024-06-01T01:52:34.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:34 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:34.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:34 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:34.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:34 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:34.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:34 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:34.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:34 smithi116 ceph-mon[25082]: from='osd.0 [v2:172.21.15.116:6802/1460215654,v1:172.21.15.116:6803/1460215654]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-01T01:52:34.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:34 smithi116 ceph-mon[25082]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-01T01:52:34.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:34 smithi116 ceph-mon[29958]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:34.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:34 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:34.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:34 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:52:34.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:34 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:34.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:34 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:52:34.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:34 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:52:34.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:34 smithi116 ceph-mon[29958]: Adjusting osd_memory_target on smithi116 to 15040M 2024-06-01T01:52:34.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:34 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:34.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:34 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:34.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:34 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:34.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:34 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:34.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:34 smithi116 ceph-mon[29958]: from='osd.0 [v2:172.21.15.116:6802/1460215654,v1:172.21.15.116:6803/1460215654]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-01T01:52:34.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:34 smithi116 ceph-mon[29958]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-01T01:52:36.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:35 smithi116 ceph-mon[29958]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-01T01:52:36.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:35 smithi116 ceph-mon[29958]: osdmap e6: 1 total, 0 up, 1 in 2024-06-01T01:52:36.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:35 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T01:52:36.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:35 smithi116 ceph-mon[29958]: from='osd.0 [v2:172.21.15.116:6802/1460215654,v1:172.21.15.116:6803/1460215654]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T01:52:36.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:35 smithi116 ceph-mon[29958]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T01:52:36.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:35 smithi116 ceph-mon[25082]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-01T01:52:36.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:35 smithi116 ceph-mon[25082]: osdmap e6: 1 total, 0 up, 1 in 2024-06-01T01:52:36.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:35 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T01:52:36.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:35 smithi116 ceph-mon[25082]: from='osd.0 [v2:172.21.15.116:6802/1460215654,v1:172.21.15.116:6803/1460215654]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T01:52:36.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:35 smithi116 ceph-mon[25082]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T01:52:36.047 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 01:52:35 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0[33260]: 2024-06-01T01:52:35.857+0000 7fab780ba700 -1 osd.0 0 waiting for initial osdmap 2024-06-01T01:52:36.047 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 01:52:35 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0[33260]: 2024-06-01T01:52:35.859+0000 7fab72250700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-01T01:52:36.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:35 smithi161 ceph-mon[26055]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-01T01:52:36.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:35 smithi161 ceph-mon[26055]: osdmap e6: 1 total, 0 up, 1 in 2024-06-01T01:52:36.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:35 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T01:52:36.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:35 smithi161 ceph-mon[26055]: from='osd.0 [v2:172.21.15.116:6802/1460215654,v1:172.21.15.116:6803/1460215654]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T01:52:36.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:35 smithi161 ceph-mon[26055]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T01:52:36.223 INFO:teuthology.orchestra.run.smithi116.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-01T01:52:36.223 INFO:teuthology.orchestra.run.smithi116.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-01T01:52:36.223 INFO:teuthology.orchestra.run.smithi116.stderr: stderr: 10+0 records in 2024-06-01T01:52:36.223 INFO:teuthology.orchestra.run.smithi116.stderr:10+0 records out 2024-06-01T01:52:36.223 INFO:teuthology.orchestra.run.smithi116.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124965 s, 839 MB/s 2024-06-01T01:52:36.223 INFO:teuthology.orchestra.run.smithi116.stderr:--> Zapping successful for: 2024-06-01T01:52:36.881 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi116:vg_nvme/lv_3 2024-06-01T01:52:37.189 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:36 smithi116 ceph-mon[25082]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:37.190 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:36 smithi116 ceph-mon[25082]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-06-01T01:52:37.190 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:36 smithi116 ceph-mon[25082]: osdmap e7: 1 total, 0 up, 1 in 2024-06-01T01:52:37.190 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:36 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T01:52:37.190 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:36 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T01:52:37.190 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:36 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T01:52:37.190 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:36 smithi116 ceph-mon[29958]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:37.190 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:36 smithi116 ceph-mon[29958]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-06-01T01:52:37.190 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:36 smithi116 ceph-mon[29958]: osdmap e7: 1 total, 0 up, 1 in 2024-06-01T01:52:37.190 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:36 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T01:52:37.190 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:36 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T01:52:37.191 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:36 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T01:52:37.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:36 smithi161 ceph-mon[26055]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:37.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:36 smithi161 ceph-mon[26055]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-06-01T01:52:37.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:36 smithi161 ceph-mon[26055]: osdmap e7: 1 total, 0 up, 1 in 2024-06-01T01:52:37.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:36 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T01:52:37.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:36 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T01:52:37.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:36 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T01:52:38.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:37 smithi161 ceph-mon[26055]: purged_snaps scrub starts 2024-06-01T01:52:38.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:37 smithi161 ceph-mon[26055]: purged_snaps scrub ok 2024-06-01T01:52:38.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:37 smithi161 ceph-mon[26055]: osd.0 [v2:172.21.15.116:6802/1460215654,v1:172.21.15.116:6803/1460215654] boot 2024-06-01T01:52:38.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:37 smithi161 ceph-mon[26055]: osdmap e8: 1 total, 1 up, 1 in 2024-06-01T01:52:38.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:37 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T01:52:38.205 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:37 smithi116 ceph-mon[29958]: purged_snaps scrub starts 2024-06-01T01:52:38.205 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:37 smithi116 ceph-mon[29958]: purged_snaps scrub ok 2024-06-01T01:52:38.205 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:37 smithi116 ceph-mon[29958]: osd.0 [v2:172.21.15.116:6802/1460215654,v1:172.21.15.116:6803/1460215654] boot 2024-06-01T01:52:38.205 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:37 smithi116 ceph-mon[29958]: osdmap e8: 1 total, 1 up, 1 in 2024-06-01T01:52:38.205 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:37 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T01:52:38.205 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:37 smithi116 ceph-mon[25082]: purged_snaps scrub starts 2024-06-01T01:52:38.205 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:37 smithi116 ceph-mon[25082]: purged_snaps scrub ok 2024-06-01T01:52:38.205 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:37 smithi116 ceph-mon[25082]: osd.0 [v2:172.21.15.116:6802/1460215654,v1:172.21.15.116:6803/1460215654] boot 2024-06-01T01:52:38.205 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:37 smithi116 ceph-mon[25082]: osdmap e8: 1 total, 1 up, 1 in 2024-06-01T01:52:38.205 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:37 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T01:52:39.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:38 smithi116 ceph-mon[25082]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:39.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:38 smithi116 ceph-mon[25082]: osdmap e9: 1 total, 1 up, 1 in 2024-06-01T01:52:39.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:38 smithi116 ceph-mon[29958]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:39.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:38 smithi116 ceph-mon[29958]: osdmap e9: 1 total, 1 up, 1 in 2024-06-01T01:52:39.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:38 smithi161 ceph-mon[26055]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-01T01:52:39.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:38 smithi161 ceph-mon[26055]: osdmap e9: 1 total, 1 up, 1 in 2024-06-01T01:52:40.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:39 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-01T01:52:40.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:39 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-01T01:52:40.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:39 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:40.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:39 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-01T01:52:40.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:39 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-01T01:52:40.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:39 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:40.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:39 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-01T01:52:40.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:39 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-01T01:52:40.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:39 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:41.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:40 smithi161 ceph-mon[26055]: from='client.24142 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:52:41.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:40 smithi161 ceph-mon[26055]: pgmap v42: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:52:41.213 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:40 smithi116 ceph-mon[25082]: from='client.24142 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:52:41.213 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:40 smithi116 ceph-mon[25082]: pgmap v42: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:52:41.214 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:40 smithi116 ceph-mon[29958]: from='client.24142 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:52:41.214 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:40 smithi116 ceph-mon[29958]: pgmap v42: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:52:41.464 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:52:41 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:52:41] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:52:42.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:42 smithi116 ceph-mon[29958]: pgmap v43: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:52:42.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:42 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:42.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:42 smithi116 ceph-mon[29958]: Detected new or changed devices on smithi116 2024-06-01T01:52:42.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:42 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:42.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:42 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/3224833625' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95343a32-622a-4716-8fcc-71ce91004700"}]: dispatch 2024-06-01T01:52:42.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:42 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/3224833625' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95343a32-622a-4716-8fcc-71ce91004700"}]': finished 2024-06-01T01:52:42.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:42 smithi116 ceph-mon[29958]: osdmap e10: 2 total, 1 up, 2 in 2024-06-01T01:52:42.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:42 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T01:52:42.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:42 smithi116 ceph-mon[25082]: pgmap v43: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:52:42.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:42 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:42.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:42 smithi116 ceph-mon[25082]: Detected new or changed devices on smithi116 2024-06-01T01:52:42.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:42 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:42.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:42 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/3224833625' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95343a32-622a-4716-8fcc-71ce91004700"}]: dispatch 2024-06-01T01:52:42.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:42 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/3224833625' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95343a32-622a-4716-8fcc-71ce91004700"}]': finished 2024-06-01T01:52:42.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:42 smithi116 ceph-mon[25082]: osdmap e10: 2 total, 1 up, 2 in 2024-06-01T01:52:42.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:42 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T01:52:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:42 smithi161 ceph-mon[26055]: pgmap v43: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:52:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:42 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:42 smithi161 ceph-mon[26055]: Detected new or changed devices on smithi116 2024-06-01T01:52:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:42 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:42 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/3224833625' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95343a32-622a-4716-8fcc-71ce91004700"}]: dispatch 2024-06-01T01:52:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:42 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/3224833625' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95343a32-622a-4716-8fcc-71ce91004700"}]': finished 2024-06-01T01:52:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:42 smithi161 ceph-mon[26055]: osdmap e10: 2 total, 1 up, 2 in 2024-06-01T01:52:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:42 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T01:52:43.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:43 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1001281814' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-01T01:52:43.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:43 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1001281814' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-01T01:52:43.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:43 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1001281814' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-01T01:52:44.514 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:44 smithi116 ceph-mon[25082]: pgmap v45: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:52:44.514 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:44 smithi116 ceph-mon[29958]: pgmap v45: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:52:44.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:44 smithi161 ceph-mon[26055]: pgmap v45: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:52:46.456 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:46 smithi116 ceph-mon[25082]: pgmap v46: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:52:46.457 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:46 smithi116 ceph-mon[29958]: pgmap v46: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:52:46.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:46 smithi161 ceph-mon[26055]: pgmap v46: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:52:48.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:48 smithi116 ceph-mon[25082]: pgmap v47: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:52:48.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:48 smithi116 ceph-mon[29958]: pgmap v47: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:52:48.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:48 smithi161 ceph-mon[26055]: pgmap v47: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:52:49.480 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:49 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-01T01:52:49.480 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:49 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:49.481 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:49 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-01T01:52:49.481 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:49 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:49.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:49 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-01T01:52:49.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:49 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:50.468 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:50 smithi116 ceph-mon[25082]: pgmap v48: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:52:50.468 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:50 smithi116 ceph-mon[25082]: Deploying daemon osd.1 on smithi116 2024-06-01T01:52:50.468 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:50 smithi116 ceph-mon[29958]: pgmap v48: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:52:50.468 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:50 smithi116 ceph-mon[29958]: Deploying daemon osd.1 on smithi116 2024-06-01T01:52:50.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:50 smithi161 ceph-mon[26055]: pgmap v48: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:52:50.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:50 smithi161 ceph-mon[26055]: Deploying daemon osd.1 on smithi116 2024-06-01T01:52:52.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:52 smithi116 ceph-mon[25082]: pgmap v49: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:52:52.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:52 smithi116 ceph-mon[29958]: pgmap v49: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:52:52.498 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:52 smithi161 ceph-mon[26055]: pgmap v49: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:52:53.975 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:52:53 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:52:53] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:52:54.875 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:54 smithi116 ceph-mon[29958]: pgmap v50: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:52:54.875 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:54 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:54.875 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:54 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:54.875 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:54 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:54.875 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:54 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:52:54.875 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:54 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:54.875 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:54 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:52:54.875 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:54 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:54.876 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:54 smithi116 ceph-mon[25082]: pgmap v50: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:52:54.876 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:54 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:54.876 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:54 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:54.876 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:54 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:54.876 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:54 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:52:54.876 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:54 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:54.876 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:54 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:52:54.876 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:54 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:55.172 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:54 smithi161 ceph-mon[26055]: pgmap v50: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:52:55.172 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:54 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:55.172 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:54 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:55.172 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:54 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:55.172 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:54 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:52:55.172 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:54 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:55.172 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:54 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:52:55.172 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:54 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:56.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:56 smithi116 ceph-mon[25082]: pgmap v51: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:52:56.969 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:56 smithi116 ceph-mon[29958]: pgmap v51: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:52:57.001 INFO:teuthology.orchestra.run.smithi116.stdout:Created osd(s) 1 on host 'smithi116' 2024-06-01T01:52:57.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:56 smithi161 ceph-mon[26055]: pgmap v51: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:52:57.595 DEBUG:teuthology.orchestra.run.smithi116:osd.1> sudo journalctl -f -n 0 -u ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.1.service 2024-06-01T01:52:57.598 INFO:tasks.cephadm:Deploying osd.2 on smithi116 with /dev/vg_nvme/lv_2... 2024-06-01T01:52:57.598 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-01T01:52:58.259 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:58 smithi116 ceph-mon[29958]: pgmap v52: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:52:58.259 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:58 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:58.259 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:58 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:52:58.259 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:58 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:58.259 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:58 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:52:58.259 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:58 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:52:58.260 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:58 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:52:58.260 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:58 smithi116 ceph-mon[29958]: Adjusting osd_memory_target on smithi116 to 7520M 2024-06-01T01:52:58.260 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:58 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:58.260 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:58 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:58.260 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:58 smithi116 ceph-mon[29958]: from='osd.1 [v2:172.21.15.116:6810/166790657,v1:172.21.15.116:6811/166790657]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-01T01:52:58.260 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:58 smithi116 ceph-mon[25082]: pgmap v52: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:52:58.260 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:58 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:58.260 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:58 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:52:58.260 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:58 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:58.260 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:58 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:52:58.260 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:58 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:52:58.260 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:58 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:52:58.261 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:58 smithi116 ceph-mon[25082]: Adjusting osd_memory_target on smithi116 to 7520M 2024-06-01T01:52:58.261 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:58 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:58.261 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:58 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:58.261 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:58 smithi116 ceph-mon[25082]: from='osd.1 [v2:172.21.15.116:6810/166790657,v1:172.21.15.116:6811/166790657]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-01T01:52:58.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:58 smithi161 ceph-mon[26055]: pgmap v52: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:52:58.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:58 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:58.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:58 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:52:58.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:58 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:52:58.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:58 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:52:58.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:58 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:52:58.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:58 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:52:58.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:58 smithi161 ceph-mon[26055]: Adjusting osd_memory_target on smithi116 to 7520M 2024-06-01T01:52:58.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:58 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:58.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:58 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:52:58.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:58 smithi161 ceph-mon[26055]: from='osd.1 [v2:172.21.15.116:6810/166790657,v1:172.21.15.116:6811/166790657]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-01T01:52:59.147 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:59 smithi116 ceph-mon[29958]: from='osd.1 [v2:172.21.15.116:6810/166790657,v1:172.21.15.116:6811/166790657]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-01T01:52:59.148 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:59 smithi116 ceph-mon[29958]: osdmap e11: 2 total, 1 up, 2 in 2024-06-01T01:52:59.148 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:59 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T01:52:59.148 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:52:59 smithi116 ceph-mon[29958]: from='osd.1 [v2:172.21.15.116:6810/166790657,v1:172.21.15.116:6811/166790657]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T01:52:59.148 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:59 smithi116 ceph-mon[25082]: from='osd.1 [v2:172.21.15.116:6810/166790657,v1:172.21.15.116:6811/166790657]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-01T01:52:59.148 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:59 smithi116 ceph-mon[25082]: osdmap e11: 2 total, 1 up, 2 in 2024-06-01T01:52:59.148 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:59 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T01:52:59.148 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:52:59 smithi116 ceph-mon[25082]: from='osd.1 [v2:172.21.15.116:6810/166790657,v1:172.21.15.116:6811/166790657]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T01:52:59.148 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 01:52:59 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1[36127]: 2024-06-01T01:52:59.026+0000 7fde243a4700 -1 osd.1 0 waiting for initial osdmap 2024-06-01T01:52:59.148 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 01:52:59 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1[36127]: 2024-06-01T01:52:59.032+0000 7fde1cd37700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-01T01:52:59.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:59 smithi161 ceph-mon[26055]: from='osd.1 [v2:172.21.15.116:6810/166790657,v1:172.21.15.116:6811/166790657]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-01T01:52:59.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:59 smithi161 ceph-mon[26055]: osdmap e11: 2 total, 1 up, 2 in 2024-06-01T01:52:59.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:59 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T01:52:59.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:52:59 smithi161 ceph-mon[26055]: from='osd.1 [v2:172.21.15.116:6810/166790657,v1:172.21.15.116:6811/166790657]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T01:52:59.628 INFO:teuthology.orchestra.run.smithi116.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-01T01:52:59.628 INFO:teuthology.orchestra.run.smithi116.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-01T01:52:59.629 INFO:teuthology.orchestra.run.smithi116.stderr: stderr: 10+0 records in 2024-06-01T01:52:59.629 INFO:teuthology.orchestra.run.smithi116.stderr:10+0 records out 2024-06-01T01:52:59.629 INFO:teuthology.orchestra.run.smithi116.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0122776 s, 854 MB/s 2024-06-01T01:52:59.629 INFO:teuthology.orchestra.run.smithi116.stderr:--> Zapping successful for: 2024-06-01T01:53:00.228 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:00 smithi116 ceph-mon[29958]: pgmap v54: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:53:00.228 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:00 smithi116 ceph-mon[29958]: from='osd.1 [v2:172.21.15.116:6810/166790657,v1:172.21.15.116:6811/166790657]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-06-01T01:53:00.228 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:00 smithi116 ceph-mon[29958]: osdmap e12: 2 total, 1 up, 2 in 2024-06-01T01:53:00.228 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:00 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T01:53:00.228 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:00 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T01:53:00.228 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:00 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T01:53:00.228 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:00 smithi116 ceph-mon[25082]: pgmap v54: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:53:00.229 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:00 smithi116 ceph-mon[25082]: from='osd.1 [v2:172.21.15.116:6810/166790657,v1:172.21.15.116:6811/166790657]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-06-01T01:53:00.229 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:00 smithi116 ceph-mon[25082]: osdmap e12: 2 total, 1 up, 2 in 2024-06-01T01:53:00.229 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:00 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T01:53:00.229 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:00 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T01:53:00.229 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:00 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T01:53:00.253 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi116:vg_nvme/lv_2 2024-06-01T01:53:00.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:00 smithi161 ceph-mon[26055]: pgmap v54: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:53:00.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:00 smithi161 ceph-mon[26055]: from='osd.1 [v2:172.21.15.116:6810/166790657,v1:172.21.15.116:6811/166790657]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-06-01T01:53:00.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:00 smithi161 ceph-mon[26055]: osdmap e12: 2 total, 1 up, 2 in 2024-06-01T01:53:00.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:00 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T01:53:00.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:00 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T01:53:00.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:00 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T01:53:01.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:01 smithi116 ceph-mon[25082]: purged_snaps scrub starts 2024-06-01T01:53:01.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:01 smithi116 ceph-mon[25082]: purged_snaps scrub ok 2024-06-01T01:53:01.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:01 smithi116 ceph-mon[25082]: osd.1 [v2:172.21.15.116:6810/166790657,v1:172.21.15.116:6811/166790657] boot 2024-06-01T01:53:01.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:01 smithi116 ceph-mon[25082]: osdmap e13: 2 total, 2 up, 2 in 2024-06-01T01:53:01.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:01 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T01:53:01.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:01 smithi116 ceph-mon[29958]: purged_snaps scrub starts 2024-06-01T01:53:01.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:01 smithi116 ceph-mon[29958]: purged_snaps scrub ok 2024-06-01T01:53:01.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:01 smithi116 ceph-mon[29958]: osd.1 [v2:172.21.15.116:6810/166790657,v1:172.21.15.116:6811/166790657] boot 2024-06-01T01:53:01.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:01 smithi116 ceph-mon[29958]: osdmap e13: 2 total, 2 up, 2 in 2024-06-01T01:53:01.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:01 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T01:53:01.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:01 smithi161 ceph-mon[26055]: purged_snaps scrub starts 2024-06-01T01:53:01.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:01 smithi161 ceph-mon[26055]: purged_snaps scrub ok 2024-06-01T01:53:01.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:01 smithi161 ceph-mon[26055]: osd.1 [v2:172.21.15.116:6810/166790657,v1:172.21.15.116:6811/166790657] boot 2024-06-01T01:53:01.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:01 smithi161 ceph-mon[26055]: osdmap e13: 2 total, 2 up, 2 in 2024-06-01T01:53:01.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:01 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T01:53:01.626 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:53:01 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:53:01] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:53:02.288 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:02 smithi116 ceph-mon[29958]: pgmap v57: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:53:02.289 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:02 smithi116 ceph-mon[29958]: osdmap e14: 2 total, 2 up, 2 in 2024-06-01T01:53:02.289 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:02 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:02.289 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:02 smithi116 ceph-mon[29958]: Detected new or changed devices on smithi116 2024-06-01T01:53:02.289 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:02 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:02.289 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:02 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:53:02.289 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:02 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:02.290 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:02 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:53:02.290 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:02 smithi116 ceph-mon[29958]: pgmap v59: 0 pgs: ; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:02.290 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:02 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:02.290 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:02 smithi116 ceph-mon[25082]: pgmap v57: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:53:02.290 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:02 smithi116 ceph-mon[25082]: osdmap e14: 2 total, 2 up, 2 in 2024-06-01T01:53:02.291 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:02 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:02.291 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:02 smithi116 ceph-mon[25082]: Detected new or changed devices on smithi116 2024-06-01T01:53:02.291 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:02 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:02.291 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:02 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:53:02.291 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:02 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:02.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:02 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:53:02.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:02 smithi116 ceph-mon[25082]: pgmap v59: 0 pgs: ; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:02.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:02 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:02.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:02 smithi161 ceph-mon[26055]: pgmap v57: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-06-01T01:53:02.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:02 smithi161 ceph-mon[26055]: osdmap e14: 2 total, 2 up, 2 in 2024-06-01T01:53:02.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:02 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:02.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:02 smithi161 ceph-mon[26055]: Detected new or changed devices on smithi116 2024-06-01T01:53:02.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:02 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:02.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:02 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:53:02.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:02 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:02.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:02 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:53:02.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:02 smithi161 ceph-mon[26055]: pgmap v59: 0 pgs: ; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:02.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:02 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:03.295 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:53:03 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:53:03] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:53:03.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:03 smithi116 ceph-mon[25082]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-06-01T01:53:03.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:03 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:03.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:03 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:03.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:03 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-01T01:53:03.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:03 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-01T01:53:03.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:03 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:03.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:03 smithi116 ceph-mon[29958]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-06-01T01:53:03.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:03 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:03.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:03 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:03.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:03 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-01T01:53:03.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:03 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-01T01:53:03.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:03 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:03.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:03 smithi161 ceph-mon[26055]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-06-01T01:53:03.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:03 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:03.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:03 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:03.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:03 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-01T01:53:03.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:03 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-01T01:53:03.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:03 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:04.318 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:04 smithi161 ceph-mon[26055]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:53:04.318 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:04 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:53:04.318 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:04 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:04.318 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:04 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:53:04.318 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:04 smithi161 ceph-mon[26055]: pgmap v60: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:04.318 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:04 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:04.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:04 smithi116 ceph-mon[25082]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:53:04.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:04 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:53:04.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:04 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:04.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:04 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:53:04.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:04 smithi116 ceph-mon[25082]: pgmap v60: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:04.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:04 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:04.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:04 smithi116 ceph-mon[29958]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:53:04.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:04 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:53:04.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:04 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:04.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:04 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:53:04.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:04 smithi116 ceph-mon[29958]: pgmap v60: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:04.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:04 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:05.377 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:05 smithi116 ceph-mon[25082]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-06-01T01:53:05.377 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:05 smithi116 ceph-mon[25082]: Cluster is now healthy 2024-06-01T01:53:05.377 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:05 smithi116 ceph-mon[29958]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-06-01T01:53:05.377 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:05 smithi116 ceph-mon[29958]: Cluster is now healthy 2024-06-01T01:53:05.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:05 smithi161 ceph-mon[26055]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-06-01T01:53:05.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:05 smithi161 ceph-mon[26055]: Cluster is now healthy 2024-06-01T01:53:06.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:06 smithi116 ceph-mon[25082]: pgmap v61: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:06.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:06 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/95354939' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d392c3f3-f08e-48d0-aa79-727eb1f9c71f"}]: dispatch 2024-06-01T01:53:06.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:06 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/95354939' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d392c3f3-f08e-48d0-aa79-727eb1f9c71f"}]': finished 2024-06-01T01:53:06.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:06 smithi116 ceph-mon[25082]: osdmap e15: 3 total, 2 up, 3 in 2024-06-01T01:53:06.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:06 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T01:53:06.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:06 smithi116 ceph-mon[29958]: pgmap v61: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:06.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:06 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/95354939' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d392c3f3-f08e-48d0-aa79-727eb1f9c71f"}]: dispatch 2024-06-01T01:53:06.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:06 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/95354939' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d392c3f3-f08e-48d0-aa79-727eb1f9c71f"}]': finished 2024-06-01T01:53:06.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:06 smithi116 ceph-mon[29958]: osdmap e15: 3 total, 2 up, 3 in 2024-06-01T01:53:06.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:06 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T01:53:06.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:06 smithi161 ceph-mon[26055]: pgmap v61: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:06.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:06 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/95354939' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d392c3f3-f08e-48d0-aa79-727eb1f9c71f"}]: dispatch 2024-06-01T01:53:06.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:06 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/95354939' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d392c3f3-f08e-48d0-aa79-727eb1f9c71f"}]': finished 2024-06-01T01:53:06.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:06 smithi161 ceph-mon[26055]: osdmap e15: 3 total, 2 up, 3 in 2024-06-01T01:53:06.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:06 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T01:53:07.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:07 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1993476449' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-01T01:53:07.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:07 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1993476449' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-01T01:53:07.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:07 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1993476449' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-01T01:53:08.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:08 smithi161 ceph-mon[26055]: pgmap v63: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:08.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:08 smithi116 ceph-mon[25082]: pgmap v63: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:08.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:08 smithi116 ceph-mon[29958]: pgmap v63: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:10.291 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:10 smithi116 ceph-mon[25082]: pgmap v64: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:10.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:10 smithi116 ceph-mon[29958]: pgmap v64: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:10.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:10 smithi161 ceph-mon[26055]: pgmap v64: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:12.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:12 smithi161 ceph-mon[26055]: pgmap v65: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:12.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:12 smithi116 ceph-mon[25082]: pgmap v65: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:12.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:12 smithi116 ceph-mon[29958]: pgmap v65: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:13.156 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:13 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T01:53:13.156 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:13 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T01:53:13.156 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:13 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T01:53:13.156 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:13 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T01:53:13.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:13 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T01:53:13.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:13 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T01:53:14.133 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:53:13 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:53:13] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:53:14.413 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:14 smithi161 ceph-mon[26055]: pgmap v66: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:14.414 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:14 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-01T01:53:14.414 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:14 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:14.414 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:14 smithi161 ceph-mon[26055]: Deploying daemon osd.2 on smithi116 2024-06-01T01:53:14.414 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:14 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:14.414 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:14 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:14.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:14 smithi116 ceph-mon[25082]: pgmap v66: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:14.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:14 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-01T01:53:14.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:14 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:14.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:14 smithi116 ceph-mon[25082]: Deploying daemon osd.2 on smithi116 2024-06-01T01:53:14.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:14 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:14.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:14 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:14.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:14 smithi116 ceph-mon[29958]: pgmap v66: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:14.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:14 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-01T01:53:14.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:14 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:14.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:14 smithi116 ceph-mon[29958]: Deploying daemon osd.2 on smithi116 2024-06-01T01:53:14.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:14 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:14.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:14 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:16.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:16 smithi116 ceph-mon[25082]: pgmap v67: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:16.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:16 smithi116 ceph-mon[29958]: pgmap v67: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:16.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:16 smithi161 ceph-mon[26055]: pgmap v67: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:18.342 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:18 smithi116 ceph-mon[25082]: pgmap v68: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:18.343 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:18 smithi116 ceph-mon[29958]: pgmap v68: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:18.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:18 smithi161 ceph-mon[26055]: pgmap v68: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:19.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:19 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:53:19.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:19 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:19.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:19 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:53:19.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:19 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:19.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:19 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:19.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:19 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:53:19.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:19 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:19.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:19 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:53:19.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:19 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:19.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:19 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:19.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:19 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:53:19.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:19 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:19.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:19 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:53:19.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:19 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:19.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:19 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:20.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:20 smithi116 ceph-mon[25082]: pgmap v69: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:20.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:20 smithi116 ceph-mon[29958]: pgmap v69: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:20.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:20 smithi161 ceph-mon[26055]: pgmap v69: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:21.202 INFO:teuthology.orchestra.run.smithi116.stdout:Created osd(s) 2 on host 'smithi116' 2024-06-01T01:53:21.861 DEBUG:teuthology.orchestra.run.smithi116:osd.2> sudo journalctl -f -n 0 -u ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.2.service 2024-06-01T01:53:21.863 INFO:tasks.cephadm:Deploying osd.3 on smithi116 with /dev/vg_nvme/lv_1... 2024-06-01T01:53:21.863 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-01T01:53:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:22 smithi161 ceph-mon[26055]: pgmap v70: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:22 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:53:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:22 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:22 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:53:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:22 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:22 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:53:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:22 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:53:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:22 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:53:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:22 smithi161 ceph-mon[26055]: Adjusting osd_memory_target on smithi116 to 5013M 2024-06-01T01:53:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:22 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:22 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:22.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:22 smithi161 ceph-mon[26055]: from='osd.2 [v2:172.21.15.116:6818/2956549862,v1:172.21.15.116:6819/2956549862]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-01T01:53:22.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:22 smithi161 ceph-mon[26055]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-01T01:53:22.459 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:22 smithi116 ceph-mon[25082]: pgmap v70: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:22.460 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:22 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:53:22.460 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:22 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:22.460 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:22 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:53:22.460 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:22 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:22.460 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:22 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:53:22.460 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:22 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:53:22.460 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:22 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:53:22.460 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:22 smithi116 ceph-mon[25082]: Adjusting osd_memory_target on smithi116 to 5013M 2024-06-01T01:53:22.460 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:22 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:22.460 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:22 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:22.460 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:22 smithi116 ceph-mon[25082]: from='osd.2 [v2:172.21.15.116:6818/2956549862,v1:172.21.15.116:6819/2956549862]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-01T01:53:22.460 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:22 smithi116 ceph-mon[25082]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-01T01:53:22.461 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:22 smithi116 ceph-mon[29958]: pgmap v70: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:22.461 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:22 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:53:22.461 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:22 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:22.461 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:22 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:53:22.461 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:22 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:22.461 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:22 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:53:22.461 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:22 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:53:22.461 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:22 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:53:22.461 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:22 smithi116 ceph-mon[29958]: Adjusting osd_memory_target on smithi116 to 5013M 2024-06-01T01:53:22.461 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:22 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:22.461 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:22 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:22.462 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:22 smithi116 ceph-mon[29958]: from='osd.2 [v2:172.21.15.116:6818/2956549862,v1:172.21.15.116:6819/2956549862]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-01T01:53:22.462 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:22 smithi116 ceph-mon[29958]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-01T01:53:23.497 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:23 smithi161 ceph-mon[26055]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-01T01:53:23.498 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:23 smithi161 ceph-mon[26055]: osdmap e16: 3 total, 2 up, 3 in 2024-06-01T01:53:23.498 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:23 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T01:53:23.498 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:23 smithi161 ceph-mon[26055]: from='osd.2 [v2:172.21.15.116:6818/2956549862,v1:172.21.15.116:6819/2956549862]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T01:53:23.498 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:23 smithi161 ceph-mon[26055]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T01:53:23.498 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:23 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:23.498 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:23 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:23.498 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:23 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:53:23.498 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:23 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:23.498 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:23 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:53:23.498 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:23 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:23.545 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:53:23 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:53:23] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:53:23.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:23 smithi116 ceph-mon[25082]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-01T01:53:23.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:23 smithi116 ceph-mon[25082]: osdmap e16: 3 total, 2 up, 3 in 2024-06-01T01:53:23.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:23 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T01:53:23.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:23 smithi116 ceph-mon[25082]: from='osd.2 [v2:172.21.15.116:6818/2956549862,v1:172.21.15.116:6819/2956549862]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T01:53:23.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:23 smithi116 ceph-mon[25082]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T01:53:23.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:23 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:23.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:23 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:23.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:23 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:53:23.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:23 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:23.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:23 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:53:23.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:23 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:23.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:23 smithi116 ceph-mon[29958]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-01T01:53:23.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:23 smithi116 ceph-mon[29958]: osdmap e16: 3 total, 2 up, 3 in 2024-06-01T01:53:23.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:23 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T01:53:23.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:23 smithi116 ceph-mon[29958]: from='osd.2 [v2:172.21.15.116:6818/2956549862,v1:172.21.15.116:6819/2956549862]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T01:53:23.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:23 smithi116 ceph-mon[29958]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T01:53:23.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:23 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:23.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:23 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:23.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:23 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:53:23.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:23 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:23.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:23 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:53:23.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:23 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:23.548 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 01:53:23 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2[39044]: 2024-06-01T01:53:23.225+0000 7f5be0f0b700 -1 osd.2 0 waiting for initial osdmap 2024-06-01T01:53:23.548 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 01:53:23 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2[39044]: 2024-06-01T01:53:23.228+0000 7f5bdd0a5700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-01T01:53:24.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:24 smithi116 ceph-mon[25082]: pgmap v72: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:24.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:24 smithi116 ceph-mon[25082]: Detected new or changed devices on smithi116 2024-06-01T01:53:24.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:24 smithi116 ceph-mon[25082]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-06-01T01:53:24.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:24 smithi116 ceph-mon[25082]: osdmap e17: 3 total, 2 up, 3 in 2024-06-01T01:53:24.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:24 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T01:53:24.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:24 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T01:53:24.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:24 smithi116 ceph-mon[29958]: pgmap v72: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:24.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:24 smithi116 ceph-mon[29958]: Detected new or changed devices on smithi116 2024-06-01T01:53:24.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:24 smithi116 ceph-mon[29958]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-06-01T01:53:24.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:24 smithi116 ceph-mon[29958]: osdmap e17: 3 total, 2 up, 3 in 2024-06-01T01:53:24.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:24 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T01:53:24.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:24 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T01:53:24.691 INFO:teuthology.orchestra.run.smithi116.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-01T01:53:24.691 INFO:teuthology.orchestra.run.smithi116.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-01T01:53:24.691 INFO:teuthology.orchestra.run.smithi116.stderr: stderr: 10+0 records in 2024-06-01T01:53:24.691 INFO:teuthology.orchestra.run.smithi116.stderr:10+0 records out 2024-06-01T01:53:24.691 INFO:teuthology.orchestra.run.smithi116.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124941 s, 839 MB/s 2024-06-01T01:53:24.691 INFO:teuthology.orchestra.run.smithi116.stderr:--> Zapping successful for: 2024-06-01T01:53:24.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:24 smithi161 ceph-mon[26055]: pgmap v72: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:24.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:24 smithi161 ceph-mon[26055]: Detected new or changed devices on smithi116 2024-06-01T01:53:24.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:24 smithi161 ceph-mon[26055]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-06-01T01:53:24.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:24 smithi161 ceph-mon[26055]: osdmap e17: 3 total, 2 up, 3 in 2024-06-01T01:53:24.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:24 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T01:53:24.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:24 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T01:53:25.015 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:53:24 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:53:24] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:53:25.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:25 smithi116 ceph-mon[29958]: purged_snaps scrub starts 2024-06-01T01:53:25.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:25 smithi116 ceph-mon[29958]: purged_snaps scrub ok 2024-06-01T01:53:25.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:25 smithi116 ceph-mon[29958]: osd.2 [v2:172.21.15.116:6818/2956549862,v1:172.21.15.116:6819/2956549862] boot 2024-06-01T01:53:25.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:25 smithi116 ceph-mon[29958]: osdmap e18: 3 total, 3 up, 3 in 2024-06-01T01:53:25.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:25 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T01:53:25.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:25 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:25.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:25 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:25.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:25 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:53:25.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:25 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:25.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:25 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:53:25.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:25 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:25.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:25 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-06-01T01:53:25.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:25 smithi116 ceph-mon[25082]: purged_snaps scrub starts 2024-06-01T01:53:25.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:25 smithi116 ceph-mon[25082]: purged_snaps scrub ok 2024-06-01T01:53:25.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:25 smithi116 ceph-mon[25082]: osd.2 [v2:172.21.15.116:6818/2956549862,v1:172.21.15.116:6819/2956549862] boot 2024-06-01T01:53:25.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:25 smithi116 ceph-mon[25082]: osdmap e18: 3 total, 3 up, 3 in 2024-06-01T01:53:25.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:25 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T01:53:25.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:25 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:25.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:25 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:25.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:25 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:53:25.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:25 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:25.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:25 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:53:25.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:25 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:25.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:25 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-06-01T01:53:25.356 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi116:vg_nvme/lv_1 2024-06-01T01:53:25.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:25 smithi161 ceph-mon[26055]: purged_snaps scrub starts 2024-06-01T01:53:25.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:25 smithi161 ceph-mon[26055]: purged_snaps scrub ok 2024-06-01T01:53:25.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:25 smithi161 ceph-mon[26055]: osd.2 [v2:172.21.15.116:6818/2956549862,v1:172.21.15.116:6819/2956549862] boot 2024-06-01T01:53:25.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:25 smithi161 ceph-mon[26055]: osdmap e18: 3 total, 3 up, 3 in 2024-06-01T01:53:25.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:25 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T01:53:25.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:25 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:25.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:25 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:25.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:25 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:53:25.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:25 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:25.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:25 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:53:25.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:25 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:25.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:25 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-06-01T01:53:26.523 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:26 smithi116 ceph-mon[29958]: pgmap v75: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:26.523 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:26 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-06-01T01:53:26.523 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:26 smithi116 ceph-mon[29958]: osdmap e19: 3 total, 3 up, 3 in 2024-06-01T01:53:26.523 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:26 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-01T01:53:26.523 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:26 smithi116 ceph-mon[25082]: pgmap v75: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:26.523 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:26 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-06-01T01:53:26.523 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:26 smithi116 ceph-mon[25082]: osdmap e19: 3 total, 3 up, 3 in 2024-06-01T01:53:26.524 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:26 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-01T01:53:26.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:26 smithi161 ceph-mon[26055]: pgmap v75: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-06-01T01:53:26.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:26 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-06-01T01:53:26.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:26 smithi161 ceph-mon[26055]: osdmap e19: 3 total, 3 up, 3 in 2024-06-01T01:53:26.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:26 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-01T01:53:27.248 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 01:53:27 smithi116 sudo[40268]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-01T01:53:27.248 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 01:53:27 smithi116 sudo[40268]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-06-01T01:53:27.248 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 01:53:27 smithi116 sudo[40268]: pam_unix(sudo:session): session closed for user root 2024-06-01T01:53:27.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:27 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-01T01:53:27.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:27 smithi116 ceph-mon[25082]: osdmap e20: 3 total, 3 up, 3 in 2024-06-01T01:53:27.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:27 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-01T01:53:27.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:27 smithi116 ceph-mon[29958]: osdmap e20: 3 total, 3 up, 3 in 2024-06-01T01:53:27.546 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 01:53:27 smithi116 sudo[40290]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-01T01:53:27.546 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 01:53:27 smithi116 sudo[40290]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-06-01T01:53:27.546 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 01:53:27 smithi116 sudo[40290]: pam_unix(sudo:session): session closed for user root 2024-06-01T01:53:27.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:27 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-01T01:53:27.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:27 smithi161 ceph-mon[26055]: osdmap e20: 3 total, 3 up, 3 in 2024-06-01T01:53:27.927 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 01:53:27 smithi116 sudo[40310]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-01T01:53:27.928 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 01:53:27 smithi116 sudo[40310]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-06-01T01:53:27.928 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 01:53:27 smithi116 sudo[40310]: pam_unix(sudo:session): session closed for user root 2024-06-01T01:53:28.205 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 01:53:27 smithi116 sudo[40319]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-01T01:53:28.205 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 01:53:27 smithi116 sudo[40319]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-06-01T01:53:28.206 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 01:53:27 smithi116 sudo[40319]: pam_unix(sudo:session): session closed for user root 2024-06-01T01:53:28.472 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 01:53:28 smithi116 sudo[40326]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-01T01:53:28.472 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 01:53:28 smithi116 sudo[40326]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-06-01T01:53:28.472 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 01:53:28 smithi116 sudo[40326]: pam_unix(sudo:session): session closed for user root 2024-06-01T01:53:28.472 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:28 smithi116 ceph-mon[25082]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:28.472 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:28 smithi116 ceph-mon[25082]: mgrmap e15: y(active, since 2m), standbys: x 2024-06-01T01:53:28.472 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:28 smithi116 ceph-mon[25082]: osdmap e21: 3 total, 3 up, 3 in 2024-06-01T01:53:28.472 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:28 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:28.473 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:28 smithi116 ceph-mon[29958]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:28.473 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:28 smithi116 ceph-mon[29958]: mgrmap e15: y(active, since 2m), standbys: x 2024-06-01T01:53:28.473 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:28 smithi116 ceph-mon[29958]: osdmap e21: 3 total, 3 up, 3 in 2024-06-01T01:53:28.473 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:28 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:28.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:28 smithi161 ceph-mon[26055]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:28.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:28 smithi161 ceph-mon[26055]: mgrmap e15: y(active, since 2m), standbys: x 2024-06-01T01:53:28.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:28 smithi161 ceph-mon[26055]: osdmap e21: 3 total, 3 up, 3 in 2024-06-01T01:53:28.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:28 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:28.723 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 01:53:28 smithi116 sudo[40351]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-01T01:53:28.724 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 01:53:28 smithi116 sudo[40351]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-06-01T01:53:28.724 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 01:53:28 smithi116 sudo[40351]: pam_unix(sudo:session): session closed for user root 2024-06-01T01:53:28.985 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:28 smithi116 sudo[40394]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-01T01:53:28.985 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:28 smithi116 sudo[40394]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-06-01T01:53:28.985 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:28 smithi116 sudo[40394]: pam_unix(sudo:session): session closed for user root 2024-06-01T01:53:29.244 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:28 smithi116 sudo[40410]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-01T01:53:29.244 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:28 smithi116 sudo[40410]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-06-01T01:53:29.244 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:28 smithi116 sudo[40410]: pam_unix(sudo:session): session closed for user root 2024-06-01T01:53:29.505 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:29 smithi116 sudo[40464]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-01T01:53:29.505 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:29 smithi116 sudo[40464]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-06-01T01:53:29.505 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:29 smithi116 sudo[40464]: pam_unix(sudo:session): session closed for user root 2024-06-01T01:53:29.760 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:29 smithi116 sudo[40494]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-01T01:53:29.760 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:29 smithi116 sudo[40494]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-06-01T01:53:29.760 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:29 smithi116 sudo[40494]: pam_unix(sudo:session): session closed for user root 2024-06-01T01:53:29.760 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:29 smithi116 ceph-mon[29958]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-01T01:53:29.760 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:29 smithi116 ceph-mon[29958]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:53:29.760 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:29 smithi116 ceph-mon[29958]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-01T01:53:29.760 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:29 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T01:53:29.760 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:29 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:53:29.761 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:29 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:53:29.761 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:29 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-01T01:53:29.761 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:29 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T01:53:29.761 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:29 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:53:29.761 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:29 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:53:29.761 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:29 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-01T01:53:29.761 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:29 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:29.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:29 smithi116 ceph-mon[25082]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-01T01:53:29.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:29 smithi116 ceph-mon[25082]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:53:29.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:29 smithi116 ceph-mon[25082]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-01T01:53:29.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:29 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T01:53:29.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:29 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:53:29.762 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:29 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:53:29.762 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:29 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-01T01:53:29.762 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:29 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T01:53:29.762 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:29 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:53:29.762 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:29 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:53:29.762 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:29 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-01T01:53:29.762 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:29 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:30.001 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:29 smithi161 sudo[28572]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-01T01:53:30.001 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:29 smithi161 sudo[28572]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-06-01T01:53:30.001 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:29 smithi161 sudo[28572]: pam_unix(sudo:session): session closed for user root 2024-06-01T01:53:30.001 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:30 smithi161 sudo[28575]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-01T01:53:30.001 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:30 smithi161 sudo[28575]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-06-01T01:53:30.001 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:30 smithi161 sudo[28575]: pam_unix(sudo:session): session closed for user root 2024-06-01T01:53:30.002 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:30 smithi161 ceph-mon[26055]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-01T01:53:30.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:30 smithi161 ceph-mon[26055]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:53:30.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:30 smithi161 ceph-mon[26055]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-01T01:53:30.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:30 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T01:53:30.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:30 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:53:30.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:30 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:53:30.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:30 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-01T01:53:30.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:30 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T01:53:30.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:30 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:53:30.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:30 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:53:30.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:30 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-01T01:53:30.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:30 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:30.746 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:30 smithi116 ceph-mon[29958]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-01T01:53:30.747 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:30 smithi116 ceph-mon[29958]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 16 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:30.747 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:30 smithi116 ceph-mon[29958]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-01T01:53:30.747 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:30 smithi116 ceph-mon[29958]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-01T01:53:30.747 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:30 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T01:53:30.747 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:30 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:53:30.747 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:30 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:53:30.747 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:30 smithi116 ceph-mon[29958]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-01T01:53:30.747 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:30 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:30.747 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:30 smithi116 ceph-mon[25082]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-01T01:53:30.747 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:30 smithi116 ceph-mon[25082]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 16 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:30.748 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:30 smithi116 ceph-mon[25082]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-01T01:53:30.748 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:30 smithi116 ceph-mon[25082]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-01T01:53:30.748 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:30 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T01:53:30.748 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:30 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:53:30.748 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:30 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:53:30.748 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:30 smithi116 ceph-mon[25082]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-01T01:53:30.748 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:30 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:30.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:30 smithi161 ceph-mon[26055]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-01T01:53:30.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:30 smithi161 ceph-mon[26055]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 16 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:30.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:30 smithi161 ceph-mon[26055]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-01T01:53:30.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:30 smithi161 ceph-mon[26055]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-01T01:53:30.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:30 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T01:53:30.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:30 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:53:30.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:30 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:53:30.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:30 smithi161 ceph-mon[26055]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-01T01:53:30.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:30 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:31.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:31 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/2535245347' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73a18daf-1e3e-476c-8375-96a67d726ef5"}]: dispatch 2024-06-01T01:53:31.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:31 smithi116 ceph-mon[25082]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73a18daf-1e3e-476c-8375-96a67d726ef5"}]: dispatch 2024-06-01T01:53:31.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:31 smithi116 ceph-mon[25082]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73a18daf-1e3e-476c-8375-96a67d726ef5"}]': finished 2024-06-01T01:53:31.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:31 smithi116 ceph-mon[25082]: osdmap e22: 4 total, 3 up, 4 in 2024-06-01T01:53:31.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:31 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T01:53:31.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:31 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/2535245347' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73a18daf-1e3e-476c-8375-96a67d726ef5"}]: dispatch 2024-06-01T01:53:31.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:31 smithi116 ceph-mon[29958]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73a18daf-1e3e-476c-8375-96a67d726ef5"}]: dispatch 2024-06-01T01:53:31.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:31 smithi116 ceph-mon[29958]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73a18daf-1e3e-476c-8375-96a67d726ef5"}]': finished 2024-06-01T01:53:31.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:31 smithi116 ceph-mon[29958]: osdmap e22: 4 total, 3 up, 4 in 2024-06-01T01:53:31.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:31 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T01:53:31.871 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:31 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/2535245347' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73a18daf-1e3e-476c-8375-96a67d726ef5"}]: dispatch 2024-06-01T01:53:31.871 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:31 smithi161 ceph-mon[26055]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73a18daf-1e3e-476c-8375-96a67d726ef5"}]: dispatch 2024-06-01T01:53:31.871 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:31 smithi161 ceph-mon[26055]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73a18daf-1e3e-476c-8375-96a67d726ef5"}]': finished 2024-06-01T01:53:31.871 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:31 smithi161 ceph-mon[26055]: osdmap e22: 4 total, 3 up, 4 in 2024-06-01T01:53:31.871 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:31 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T01:53:32.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:32 smithi116 ceph-mon[25082]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:32.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:32 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1774220314' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-01T01:53:32.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:32 smithi116 ceph-mon[29958]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:32.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:32 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1774220314' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-01T01:53:32.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:32 smithi161 ceph-mon[26055]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:32.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:32 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1774220314' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-01T01:53:34.295 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:53:33 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:53:33] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:53:34.660 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:34 smithi116 ceph-mon[25082]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:34.661 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:34 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:34.661 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:34 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:34.661 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:34 smithi116 ceph-mon[29958]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:34.661 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:34 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:34.661 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:34 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:34.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:34 smithi161 ceph-mon[26055]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:34.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:34 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:34.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:34 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:36.698 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:36 smithi116 ceph-mon[25082]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:36.698 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:36 smithi116 ceph-mon[29958]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:36.898 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:36 smithi161 ceph-mon[26055]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:38.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:38 smithi116 ceph-mon[25082]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:38.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:38 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-01T01:53:38.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:38 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:38.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:38 smithi116 ceph-mon[25082]: Deploying daemon osd.3 on smithi116 2024-06-01T01:53:38.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:38 smithi116 ceph-mon[29958]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:38.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:38 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-01T01:53:38.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:38 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:38.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:38 smithi116 ceph-mon[29958]: Deploying daemon osd.3 on smithi116 2024-06-01T01:53:38.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:38 smithi161 ceph-mon[26055]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:38.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:38 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-01T01:53:38.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:38 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:38.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:38 smithi161 ceph-mon[26055]: Deploying daemon osd.3 on smithi116 2024-06-01T01:53:40.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:40 smithi116 ceph-mon[25082]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:40.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:40 smithi116 ceph-mon[29958]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:40.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:40 smithi161 ceph-mon[26055]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:42.356 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:42 smithi116 ceph-mon[25082]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:42.356 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:42 smithi116 ceph-mon[29958]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:42.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:42 smithi161 ceph-mon[26055]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:44.904 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:53:44 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:53:44] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:53:44.904 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:44 smithi116 ceph-mon[25082]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:44.904 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:44 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:44.904 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:44 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:53:44.905 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:44 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:44.905 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:44 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:53:44.905 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:44 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:44.905 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:44 smithi116 ceph-mon[29958]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:44.905 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:44 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:44.905 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:44 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:53:44.905 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:44 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:44.905 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:44 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:53:44.905 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:44 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:44 smithi161 ceph-mon[26055]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:44 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:44 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:53:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:44 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:44.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:44 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:53:44.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:44 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:45.650 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:53:45 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:53:45] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:53:45.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:45 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:45.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:45 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:45.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:45 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:53:45.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:45 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:45 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:53:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:45 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:45 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:45 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:45 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:53:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:45 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:45 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:53:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:45 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:46.027 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:45 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:46.027 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:45 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:46.028 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:45 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:53:46.028 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:45 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:46.028 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:45 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:53:46.028 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:45 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:46.028 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:45 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:46.028 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:45 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:46.028 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:45 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:53:46.028 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:45 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:46.028 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:45 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:53:46.028 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:45 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:46.028 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:45 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:46.028 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:45 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:46.029 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:45 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:53:46.029 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:45 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:46.029 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:45 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:53:46.029 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:45 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:46.029 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:45 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:46.029 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:45 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:46.029 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:45 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:53:46.029 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:45 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:46.029 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:45 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:53:46.029 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:45 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:46.900 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:46 smithi116 ceph-mon[25082]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:46.900 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:46 smithi116 ceph-mon[29958]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:47.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:46 smithi161 ceph-mon[26055]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:47.625 INFO:teuthology.orchestra.run.smithi116.stdout:Created osd(s) 3 on host 'smithi116' 2024-06-01T01:53:47.956 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:47 smithi116 ceph-mon[29958]: from='osd.3 [v2:172.21.15.116:6826/3228322598,v1:172.21.15.116:6827/3228322598]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-01T01:53:47.956 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:47 smithi116 ceph-mon[29958]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-01T01:53:47.956 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:47 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:47.956 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:47 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:53:47.956 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:47 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:47.956 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:47 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:53:47.956 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:47 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:53:47.956 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:47 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:53:47.956 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:47 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:53:47.956 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:47 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:53:47.957 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:47 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:47.957 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:47 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:47.957 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:47 smithi116 ceph-mon[25082]: from='osd.3 [v2:172.21.15.116:6826/3228322598,v1:172.21.15.116:6827/3228322598]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-01T01:53:47.957 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:47 smithi116 ceph-mon[25082]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-01T01:53:47.957 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:47 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:47.957 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:47 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:53:47.957 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:47 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:47.957 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:47 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:53:47.957 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:47 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:53:47.957 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:47 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:53:47.957 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:47 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:53:47.958 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:47 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:53:47.958 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:47 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:47.958 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:47 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:48.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:47 smithi161 ceph-mon[26055]: from='osd.3 [v2:172.21.15.116:6826/3228322598,v1:172.21.15.116:6827/3228322598]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-01T01:53:48.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:47 smithi161 ceph-mon[26055]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-01T01:53:48.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:47 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:48.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:47 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:53:48.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:47 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:48.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:47 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:53:48.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:47 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:53:48.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:47 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:53:48.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:47 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:53:48.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:47 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:53:48.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:47 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:48.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:47 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:48.286 DEBUG:teuthology.orchestra.run.smithi116:osd.3> sudo journalctl -f -n 0 -u ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.3.service 2024-06-01T01:53:48.289 INFO:tasks.cephadm:Deploying osd.4 on smithi161 with /dev/vg_nvme/lv_4... 2024-06-01T01:53:48.289 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-01T01:53:48.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:48 smithi116 ceph-mon[29958]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:48.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:48 smithi116 ceph-mon[29958]: Adjusting osd_memory_target on smithi116 to 3760M 2024-06-01T01:53:48.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:48 smithi116 ceph-mon[29958]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-01T01:53:48.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:48 smithi116 ceph-mon[29958]: osdmap e23: 4 total, 3 up, 4 in 2024-06-01T01:53:48.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:48 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T01:53:48.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:48 smithi116 ceph-mon[29958]: from='osd.3 [v2:172.21.15.116:6826/3228322598,v1:172.21.15.116:6827/3228322598]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T01:53:48.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:48 smithi116 ceph-mon[29958]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T01:53:48.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:48 smithi116 ceph-mon[25082]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:48.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:48 smithi116 ceph-mon[25082]: Adjusting osd_memory_target on smithi116 to 3760M 2024-06-01T01:53:48.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:48 smithi116 ceph-mon[25082]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-01T01:53:48.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:48 smithi116 ceph-mon[25082]: osdmap e23: 4 total, 3 up, 4 in 2024-06-01T01:53:48.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:48 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T01:53:48.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:48 smithi116 ceph-mon[25082]: from='osd.3 [v2:172.21.15.116:6826/3228322598,v1:172.21.15.116:6827/3228322598]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T01:53:48.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:48 smithi116 ceph-mon[25082]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T01:53:48.797 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 01:53:48 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3[42191]: 2024-06-01T01:53:48.717+0000 7f0d00b2e700 -1 osd.3 0 waiting for initial osdmap 2024-06-01T01:53:48.797 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 01:53:48 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3[42191]: 2024-06-01T01:53:48.721+0000 7f0cfb4c5700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-01T01:53:48.916 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:48 smithi161 ceph-mon[26055]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:48.916 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:48 smithi161 ceph-mon[26055]: Adjusting osd_memory_target on smithi116 to 3760M 2024-06-01T01:53:48.917 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:48 smithi161 ceph-mon[26055]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-01T01:53:48.917 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:48 smithi161 ceph-mon[26055]: osdmap e23: 4 total, 3 up, 4 in 2024-06-01T01:53:48.917 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:48 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T01:53:48.917 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:48 smithi161 ceph-mon[26055]: from='osd.3 [v2:172.21.15.116:6826/3228322598,v1:172.21.15.116:6827/3228322598]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T01:53:48.917 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:48 smithi161 ceph-mon[26055]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T01:53:49.799 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:49 smithi161 ceph-mon[26055]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-06-01T01:53:49.799 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:49 smithi161 ceph-mon[26055]: osdmap e24: 4 total, 3 up, 4 in 2024-06-01T01:53:49.799 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:49 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T01:53:49.799 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:49 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T01:53:49.996 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:49 smithi116 ceph-mon[29958]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-06-01T01:53:49.996 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:49 smithi116 ceph-mon[29958]: osdmap e24: 4 total, 3 up, 4 in 2024-06-01T01:53:49.997 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:49 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T01:53:49.997 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:49 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T01:53:49.997 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:49 smithi116 ceph-mon[25082]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-06-01T01:53:49.997 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:49 smithi116 ceph-mon[25082]: osdmap e24: 4 total, 3 up, 4 in 2024-06-01T01:53:49.997 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:49 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T01:53:49.997 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:49 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T01:53:50.274 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-01T01:53:50.274 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-01T01:53:50.274 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-06-01T01:53:50.274 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-06-01T01:53:50.274 INFO:teuthology.orchestra.run.smithi161.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114382 s, 917 MB/s 2024-06-01T01:53:50.274 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-06-01T01:53:50.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:50 smithi161 ceph-mon[26055]: purged_snaps scrub starts 2024-06-01T01:53:50.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:50 smithi161 ceph-mon[26055]: purged_snaps scrub ok 2024-06-01T01:53:50.907 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:50 smithi161 ceph-mon[26055]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:50.907 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:50 smithi161 ceph-mon[26055]: osd.3 [v2:172.21.15.116:6826/3228322598,v1:172.21.15.116:6827/3228322598] boot 2024-06-01T01:53:50.907 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:50 smithi161 ceph-mon[26055]: osdmap e25: 4 total, 4 up, 4 in 2024-06-01T01:53:50.907 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:50 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T01:53:50.908 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_4 2024-06-01T01:53:51.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:50 smithi116 ceph-mon[25082]: purged_snaps scrub starts 2024-06-01T01:53:51.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:50 smithi116 ceph-mon[25082]: purged_snaps scrub ok 2024-06-01T01:53:51.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:50 smithi116 ceph-mon[25082]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:51.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:50 smithi116 ceph-mon[25082]: osd.3 [v2:172.21.15.116:6826/3228322598,v1:172.21.15.116:6827/3228322598] boot 2024-06-01T01:53:51.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:50 smithi116 ceph-mon[25082]: osdmap e25: 4 total, 4 up, 4 in 2024-06-01T01:53:51.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:50 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T01:53:51.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:50 smithi116 ceph-mon[29958]: purged_snaps scrub starts 2024-06-01T01:53:51.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:50 smithi116 ceph-mon[29958]: purged_snaps scrub ok 2024-06-01T01:53:51.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:50 smithi116 ceph-mon[29958]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-06-01T01:53:51.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:50 smithi116 ceph-mon[29958]: osd.3 [v2:172.21.15.116:6826/3228322598,v1:172.21.15.116:6827/3228322598] boot 2024-06-01T01:53:51.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:50 smithi116 ceph-mon[29958]: osdmap e25: 4 total, 4 up, 4 in 2024-06-01T01:53:51.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:50 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T01:53:52.124 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:52 smithi161 ceph-mon[26055]: osdmap e26: 4 total, 4 up, 4 in 2024-06-01T01:53:52.124 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:52 smithi161 ceph-mon[26055]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:53:52.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:52 smithi116 ceph-mon[25082]: osdmap e26: 4 total, 4 up, 4 in 2024-06-01T01:53:52.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:52 smithi116 ceph-mon[25082]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:53:52.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:52 smithi116 ceph-mon[29958]: osdmap e26: 4 total, 4 up, 4 in 2024-06-01T01:53:52.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:52 smithi116 ceph-mon[29958]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:53:53.158 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:53 smithi161 ceph-mon[26055]: from='client.24197 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:53:53.158 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:53 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-01T01:53:53.158 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:53 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-01T01:53:53.158 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:53 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:53.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:53 smithi116 ceph-mon[25082]: from='client.24197 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:53:53.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:53 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-01T01:53:53.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:53 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-01T01:53:53.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:53 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:53.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:53 smithi116 ceph-mon[29958]: from='client.24197 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:53:53.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:53 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-01T01:53:53.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:53 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-01T01:53:53.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:53 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:53:54.142 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:54 smithi161 ceph-mon[26055]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:53:54.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:54 smithi116 ceph-mon[29958]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:53:54.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:54 smithi116 ceph-mon[25082]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:53:54.296 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:53:54 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:53:54] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:53:55.390 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:55 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:55.391 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:55 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:55.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:55 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:55.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:55 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:55.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:55 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:55.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:55 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:53:56.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:56 smithi161 ceph-mon[26055]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:53:56.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:56 smithi161 ceph-mon[26055]: from='client.? 172.21.15.161:0/3030876710' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b1b3903-2ee8-4483-aeac-e6ffb3eff7d1"}]: dispatch 2024-06-01T01:53:56.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:56 smithi161 ceph-mon[26055]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b1b3903-2ee8-4483-aeac-e6ffb3eff7d1"}]: dispatch 2024-06-01T01:53:56.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:56 smithi161 ceph-mon[26055]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b1b3903-2ee8-4483-aeac-e6ffb3eff7d1"}]': finished 2024-06-01T01:53:56.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:56 smithi161 ceph-mon[26055]: osdmap e27: 5 total, 4 up, 5 in 2024-06-01T01:53:56.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:56 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T01:53:56.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:56 smithi161 ceph-mon[26055]: from='client.? 172.21.15.161:0/57960885' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-01T01:53:56.346 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:56 smithi116 ceph-mon[25082]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:53:56.347 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:56 smithi116 ceph-mon[25082]: from='client.? 172.21.15.161:0/3030876710' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b1b3903-2ee8-4483-aeac-e6ffb3eff7d1"}]: dispatch 2024-06-01T01:53:56.347 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:56 smithi116 ceph-mon[25082]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b1b3903-2ee8-4483-aeac-e6ffb3eff7d1"}]: dispatch 2024-06-01T01:53:56.347 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:56 smithi116 ceph-mon[25082]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b1b3903-2ee8-4483-aeac-e6ffb3eff7d1"}]': finished 2024-06-01T01:53:56.347 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:56 smithi116 ceph-mon[25082]: osdmap e27: 5 total, 4 up, 5 in 2024-06-01T01:53:56.347 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:56 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T01:53:56.347 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:56 smithi116 ceph-mon[25082]: from='client.? 172.21.15.161:0/57960885' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-01T01:53:56.347 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:56 smithi116 ceph-mon[29958]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:53:56.347 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:56 smithi116 ceph-mon[29958]: from='client.? 172.21.15.161:0/3030876710' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b1b3903-2ee8-4483-aeac-e6ffb3eff7d1"}]: dispatch 2024-06-01T01:53:56.347 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:56 smithi116 ceph-mon[29958]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b1b3903-2ee8-4483-aeac-e6ffb3eff7d1"}]: dispatch 2024-06-01T01:53:56.347 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:56 smithi116 ceph-mon[29958]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b1b3903-2ee8-4483-aeac-e6ffb3eff7d1"}]': finished 2024-06-01T01:53:56.347 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:56 smithi116 ceph-mon[29958]: osdmap e27: 5 total, 4 up, 5 in 2024-06-01T01:53:56.348 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:56 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T01:53:56.348 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:56 smithi116 ceph-mon[29958]: from='client.? 172.21.15.161:0/57960885' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-01T01:53:58.422 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:53:58 smithi161 ceph-mon[26055]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:53:58.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:53:58 smithi116 ceph-mon[25082]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:53:58.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:53:58 smithi116 ceph-mon[29958]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:54:00.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:00 smithi116 ceph-mon[25082]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:54:00.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:00 smithi116 ceph-mon[29958]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:54:00.405 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:00 smithi161 ceph-mon[26055]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:54:02.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:02 smithi161 ceph-mon[26055]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:54:02.517 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:02 smithi116 ceph-mon[25082]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:54:02.517 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:02 smithi116 ceph-mon[29958]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:54:02.795 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:54:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:54:02] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:54:03.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:03 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-01T01:54:03.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:03 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:03.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:03 smithi161 ceph-mon[26055]: Deploying daemon osd.4 on smithi161 2024-06-01T01:54:03.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:03 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:03.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:03 smithi161 ceph-mon[26055]: Detected new or changed devices on smithi116 2024-06-01T01:54:03.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:03 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:03.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:03 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-01T01:54:03.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:03 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:03.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:03 smithi116 ceph-mon[25082]: Deploying daemon osd.4 on smithi161 2024-06-01T01:54:03.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:03 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:03.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:03 smithi116 ceph-mon[25082]: Detected new or changed devices on smithi116 2024-06-01T01:54:03.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:03 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:03.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:03 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-01T01:54:03.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:03 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:03.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:03 smithi116 ceph-mon[29958]: Deploying daemon osd.4 on smithi161 2024-06-01T01:54:03.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:03 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:03.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:03 smithi116 ceph-mon[29958]: Detected new or changed devices on smithi116 2024-06-01T01:54:03.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:03 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:04.734 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:04 smithi161 ceph-mon[26055]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:54:04.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:04 smithi116 ceph-mon[29958]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:54:04.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:04 smithi116 ceph-mon[25082]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:54:06.646 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:06 smithi161 ceph-mon[26055]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:54:06.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:06 smithi116 ceph-mon[25082]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:54:06.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:06 smithi116 ceph-mon[29958]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:54:08.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:08 smithi116 ceph-mon[25082]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:54:08.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:08 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:08.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:08 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:08.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:08 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:08.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:08 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:08.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:08 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:08.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:08 smithi116 ceph-mon[29958]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:54:08.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:08 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:08.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:08 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:08.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:08 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:08.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:08 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:08.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:08 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:08.555 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:08 smithi161 ceph-mon[26055]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:54:08.555 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:08 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:08.555 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:08 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:08.555 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:08 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:08.555 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:08 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:08.555 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:08 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:09.925 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 4 on host 'smithi161' 2024-06-01T01:54:10.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:10 smithi161 ceph-mon[26055]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:54:10.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:10 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:10.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:10 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:10.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:10 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:10.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:10 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:10.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:10 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:54:10.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:10 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:10.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:10 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:10.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:10 smithi116 ceph-mon[25082]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:54:10.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:10 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:10.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:10 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:10.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:10 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:10.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:10 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:10.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:10 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:54:10.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:10 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:10.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:10 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:10.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:10 smithi116 ceph-mon[29958]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:54:10.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:10 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:10.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:10 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:10.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:10 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:10.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:10 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:10.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:10 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:54:10.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:10 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:10.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:10 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:10.570 DEBUG:teuthology.orchestra.run.smithi161:osd.4> sudo journalctl -f -n 0 -u ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.4.service 2024-06-01T01:54:10.573 INFO:tasks.cephadm:Deploying osd.5 on smithi161 with /dev/vg_nvme/lv_3... 2024-06-01T01:54:10.573 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-01T01:54:11.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:11 smithi161 ceph-mon[26055]: Adjusting osd_memory_target on smithi161 to 16064M 2024-06-01T01:54:11.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:11 smithi161 ceph-mon[26055]: from='osd.4 [v2:172.21.15.161:6800/147846900,v1:172.21.15.161:6801/147846900]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-01T01:54:11.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:11 smithi161 ceph-mon[26055]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-01T01:54:11.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:11 smithi116 ceph-mon[25082]: Adjusting osd_memory_target on smithi161 to 16064M 2024-06-01T01:54:11.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:11 smithi116 ceph-mon[25082]: from='osd.4 [v2:172.21.15.161:6800/147846900,v1:172.21.15.161:6801/147846900]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-01T01:54:11.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:11 smithi116 ceph-mon[25082]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-01T01:54:11.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:11 smithi116 ceph-mon[29958]: Adjusting osd_memory_target on smithi161 to 16064M 2024-06-01T01:54:11.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:11 smithi116 ceph-mon[29958]: from='osd.4 [v2:172.21.15.161:6800/147846900,v1:172.21.15.161:6801/147846900]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-01T01:54:11.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:11 smithi116 ceph-mon[29958]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-01T01:54:12.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:12 smithi161 ceph-mon[26055]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-01T01:54:12.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:12 smithi161 ceph-mon[26055]: osdmap e28: 5 total, 4 up, 5 in 2024-06-01T01:54:12.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:12 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T01:54:12.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:12 smithi161 ceph-mon[26055]: from='osd.4 [v2:172.21.15.161:6800/147846900,v1:172.21.15.161:6801/147846900]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T01:54:12.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:12 smithi161 ceph-mon[26055]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T01:54:12.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:12 smithi161 ceph-mon[26055]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:54:12.101 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 01:54:12 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[30761]: 2024-06-01T01:54:12.055+0000 7fc4bb8d7700 -1 osd.4 0 waiting for initial osdmap 2024-06-01T01:54:12.101 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 01:54:12 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[30761]: 2024-06-01T01:54:12.060+0000 7fc4b426a700 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-01T01:54:12.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:12 smithi116 ceph-mon[25082]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-01T01:54:12.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:12 smithi116 ceph-mon[25082]: osdmap e28: 5 total, 4 up, 5 in 2024-06-01T01:54:12.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:12 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T01:54:12.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:12 smithi116 ceph-mon[25082]: from='osd.4 [v2:172.21.15.161:6800/147846900,v1:172.21.15.161:6801/147846900]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T01:54:12.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:12 smithi116 ceph-mon[25082]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T01:54:12.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:12 smithi116 ceph-mon[25082]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:54:12.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:12 smithi116 ceph-mon[29958]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-01T01:54:12.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:12 smithi116 ceph-mon[29958]: osdmap e28: 5 total, 4 up, 5 in 2024-06-01T01:54:12.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:12 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T01:54:12.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:12 smithi116 ceph-mon[29958]: from='osd.4 [v2:172.21.15.161:6800/147846900,v1:172.21.15.161:6801/147846900]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T01:54:12.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:12 smithi116 ceph-mon[29958]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T01:54:12.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:12 smithi116 ceph-mon[29958]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-06-01T01:54:12.594 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-01T01:54:12.594 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-01T01:54:12.594 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-06-01T01:54:12.594 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-06-01T01:54:12.594 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124209 s, 844 MB/s 2024-06-01T01:54:12.594 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-06-01T01:54:13.113 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_3 2024-06-01T01:54:13.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:13 smithi161 ceph-mon[26055]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-06-01T01:54:13.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:13 smithi161 ceph-mon[26055]: osdmap e29: 5 total, 4 up, 5 in 2024-06-01T01:54:13.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:13 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T01:54:13.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:13 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T01:54:13.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:13 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T01:54:13.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:13 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T01:54:13.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:13 smithi116 ceph-mon[25082]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-06-01T01:54:13.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:13 smithi116 ceph-mon[25082]: osdmap e29: 5 total, 4 up, 5 in 2024-06-01T01:54:13.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:13 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T01:54:13.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:13 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T01:54:13.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:13 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T01:54:13.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:13 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T01:54:13.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:13 smithi116 ceph-mon[29958]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-06-01T01:54:13.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:13 smithi116 ceph-mon[29958]: osdmap e29: 5 total, 4 up, 5 in 2024-06-01T01:54:13.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:13 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T01:54:13.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:13 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T01:54:13.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:13 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T01:54:13.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:13 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T01:54:14.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:14 smithi161 ceph-mon[26055]: purged_snaps scrub starts 2024-06-01T01:54:14.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:14 smithi161 ceph-mon[26055]: purged_snaps scrub ok 2024-06-01T01:54:14.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:14 smithi161 ceph-mon[26055]: osd.4 [v2:172.21.15.161:6800/147846900,v1:172.21.15.161:6801/147846900] boot 2024-06-01T01:54:14.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:14 smithi161 ceph-mon[26055]: osdmap e30: 5 total, 5 up, 5 in 2024-06-01T01:54:14.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:14 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T01:54:14.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:14 smithi161 ceph-mon[26055]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2024-06-01T01:54:14.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:14 smithi161 ceph-mon[26055]: osdmap e31: 5 total, 5 up, 5 in 2024-06-01T01:54:14.545 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:54:14 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:54:14] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:54:14.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:14 smithi116 ceph-mon[25082]: purged_snaps scrub starts 2024-06-01T01:54:14.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:14 smithi116 ceph-mon[25082]: purged_snaps scrub ok 2024-06-01T01:54:14.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:14 smithi116 ceph-mon[25082]: osd.4 [v2:172.21.15.161:6800/147846900,v1:172.21.15.161:6801/147846900] boot 2024-06-01T01:54:14.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:14 smithi116 ceph-mon[25082]: osdmap e30: 5 total, 5 up, 5 in 2024-06-01T01:54:14.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:14 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T01:54:14.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:14 smithi116 ceph-mon[25082]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2024-06-01T01:54:14.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:14 smithi116 ceph-mon[25082]: osdmap e31: 5 total, 5 up, 5 in 2024-06-01T01:54:14.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:14 smithi116 ceph-mon[29958]: purged_snaps scrub starts 2024-06-01T01:54:14.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:14 smithi116 ceph-mon[29958]: purged_snaps scrub ok 2024-06-01T01:54:14.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:14 smithi116 ceph-mon[29958]: osd.4 [v2:172.21.15.161:6800/147846900,v1:172.21.15.161:6801/147846900] boot 2024-06-01T01:54:14.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:14 smithi116 ceph-mon[29958]: osdmap e30: 5 total, 5 up, 5 in 2024-06-01T01:54:14.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:14 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T01:54:14.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:14 smithi116 ceph-mon[29958]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2024-06-01T01:54:14.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:14 smithi116 ceph-mon[29958]: osdmap e31: 5 total, 5 up, 5 in 2024-06-01T01:54:15.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:15 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:15.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:15 smithi161 ceph-mon[26055]: Detected new or changed devices on smithi161 2024-06-01T01:54:15.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:15 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:15.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:15 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:15.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:15 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:15.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:15 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:15.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:15 smithi161 ceph-mon[26055]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2024-06-01T01:54:15.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:15 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:15.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:15 smithi161 ceph-mon[26055]: osdmap e32: 5 total, 5 up, 5 in 2024-06-01T01:54:15.447 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:15 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:15.447 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:15 smithi116 ceph-mon[29958]: Detected new or changed devices on smithi161 2024-06-01T01:54:15.447 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:15 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:15.447 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:15 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:15.447 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:15 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:15.448 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:15 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:15.448 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:15 smithi116 ceph-mon[29958]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2024-06-01T01:54:15.448 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:15 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:15.448 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:15 smithi116 ceph-mon[29958]: osdmap e32: 5 total, 5 up, 5 in 2024-06-01T01:54:15.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:15 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:15.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:15 smithi116 ceph-mon[25082]: Detected new or changed devices on smithi161 2024-06-01T01:54:15.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:15 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:15.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:15 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:15.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:15 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:15.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:15 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:15.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:15 smithi116 ceph-mon[25082]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2024-06-01T01:54:15.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:15 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:15.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:15 smithi116 ceph-mon[25082]: osdmap e32: 5 total, 5 up, 5 in 2024-06-01T01:54:15.795 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:54:15 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:54:15] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:54:16.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:16 smithi161 ceph-mon[26055]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-06-01T01:54:16.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:16 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:16.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:16 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:16.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:16 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:16.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:16 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:16.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:16 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:16.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:16 smithi161 ceph-mon[26055]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2024-06-01T01:54:16.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:16 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:16.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:16 smithi161 ceph-mon[26055]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:54:16.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:16 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-01T01:54:16.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:16 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-01T01:54:16.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:16 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:16.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:16 smithi161 ceph-mon[26055]: osdmap e33: 5 total, 5 up, 5 in 2024-06-01T01:54:16.458 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:16 smithi116 ceph-mon[29958]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-06-01T01:54:16.458 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:16 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:16.458 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:16 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:16.458 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:16 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:16.459 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:16 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:16.459 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:16 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:16.459 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:16 smithi116 ceph-mon[29958]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2024-06-01T01:54:16.459 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:16 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:16.459 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:16 smithi116 ceph-mon[29958]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:54:16.459 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:16 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-01T01:54:16.459 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:16 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-01T01:54:16.459 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:16 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:16.459 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:16 smithi116 ceph-mon[29958]: osdmap e33: 5 total, 5 up, 5 in 2024-06-01T01:54:16.459 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:16 smithi116 ceph-mon[25082]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-06-01T01:54:16.459 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:16 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:16.460 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:16 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:16.460 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:16 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:16.460 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:16 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:16.460 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:16 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:16.460 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:16 smithi116 ceph-mon[25082]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2024-06-01T01:54:16.460 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:16 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:16.460 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:16 smithi116 ceph-mon[25082]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:54:16.460 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:16 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-01T01:54:16.460 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:16 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-01T01:54:16.460 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:16 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:16.460 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:16 smithi116 ceph-mon[25082]: osdmap e33: 5 total, 5 up, 5 in 2024-06-01T01:54:17.315 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:17 smithi161 ceph-mon[26055]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-06-01T01:54:17.315 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:17 smithi161 ceph-mon[26055]: Cluster is now healthy 2024-06-01T01:54:17.315 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:17 smithi161 ceph-mon[26055]: osdmap e34: 5 total, 5 up, 5 in 2024-06-01T01:54:17.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:17 smithi116 ceph-mon[29958]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-06-01T01:54:17.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:17 smithi116 ceph-mon[29958]: Cluster is now healthy 2024-06-01T01:54:17.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:17 smithi116 ceph-mon[29958]: osdmap e34: 5 total, 5 up, 5 in 2024-06-01T01:54:17.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:17 smithi116 ceph-mon[25082]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-06-01T01:54:17.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:17 smithi116 ceph-mon[25082]: Cluster is now healthy 2024-06-01T01:54:17.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:17 smithi116 ceph-mon[25082]: osdmap e34: 5 total, 5 up, 5 in 2024-06-01T01:54:18.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:18 smithi161 ceph-mon[26055]: pgmap v118: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-06-01T01:54:18.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:18 smithi161 ceph-mon[26055]: from='client.? 172.21.15.161:0/2194926435' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f14db41-92df-409b-9fbd-070185f04e0c"}]: dispatch 2024-06-01T01:54:18.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:18 smithi161 ceph-mon[26055]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f14db41-92df-409b-9fbd-070185f04e0c"}]: dispatch 2024-06-01T01:54:18.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:18 smithi161 ceph-mon[26055]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f14db41-92df-409b-9fbd-070185f04e0c"}]': finished 2024-06-01T01:54:18.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:18 smithi161 ceph-mon[26055]: osdmap e35: 6 total, 5 up, 6 in 2024-06-01T01:54:18.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:18 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T01:54:18.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:18 smithi116 ceph-mon[25082]: pgmap v118: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-06-01T01:54:18.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:18 smithi116 ceph-mon[25082]: from='client.? 172.21.15.161:0/2194926435' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f14db41-92df-409b-9fbd-070185f04e0c"}]: dispatch 2024-06-01T01:54:18.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:18 smithi116 ceph-mon[25082]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f14db41-92df-409b-9fbd-070185f04e0c"}]: dispatch 2024-06-01T01:54:18.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:18 smithi116 ceph-mon[25082]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f14db41-92df-409b-9fbd-070185f04e0c"}]': finished 2024-06-01T01:54:18.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:18 smithi116 ceph-mon[25082]: osdmap e35: 6 total, 5 up, 6 in 2024-06-01T01:54:18.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:18 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T01:54:18.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:18 smithi116 ceph-mon[29958]: pgmap v118: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-06-01T01:54:18.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:18 smithi116 ceph-mon[29958]: from='client.? 172.21.15.161:0/2194926435' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f14db41-92df-409b-9fbd-070185f04e0c"}]: dispatch 2024-06-01T01:54:18.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:18 smithi116 ceph-mon[29958]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f14db41-92df-409b-9fbd-070185f04e0c"}]: dispatch 2024-06-01T01:54:18.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:18 smithi116 ceph-mon[29958]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f14db41-92df-409b-9fbd-070185f04e0c"}]': finished 2024-06-01T01:54:18.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:18 smithi116 ceph-mon[29958]: osdmap e35: 6 total, 5 up, 6 in 2024-06-01T01:54:18.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:18 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T01:54:19.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:19 smithi161 ceph-mon[26055]: from='client.? 172.21.15.161:0/3152389368' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-01T01:54:19.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:19 smithi116 ceph-mon[25082]: from='client.? 172.21.15.161:0/3152389368' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-01T01:54:19.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:19 smithi116 ceph-mon[29958]: from='client.? 172.21.15.161:0/3152389368' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-01T01:54:20.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:20 smithi161 ceph-mon[26055]: pgmap v120: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-06-01T01:54:20.529 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:20 smithi116 ceph-mon[25082]: pgmap v120: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-06-01T01:54:20.530 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:20 smithi116 ceph-mon[29958]: pgmap v120: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-06-01T01:54:22.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:22 smithi161 ceph-mon[26055]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-06-01T01:54:22.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:22 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:22.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:22 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:23.045 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:54:22 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:54:22] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:54:23.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:22 smithi116 ceph-mon[29958]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-06-01T01:54:23.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:22 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:23.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:22 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:23.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:22 smithi116 ceph-mon[25082]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-06-01T01:54:23.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:22 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:23.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:22 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:24.980 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:24 smithi161 ceph-mon[26055]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-06-01T01:54:25.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:24 smithi116 ceph-mon[25082]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-06-01T01:54:25.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:24 smithi116 ceph-mon[29958]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-06-01T01:54:25.932 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:25 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-01T01:54:25.932 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:25 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:25.932 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:25 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-01T01:54:25.932 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:25 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:26.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:25 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-01T01:54:26.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:25 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:26.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:26 smithi161 ceph-mon[26055]: Deploying daemon osd.5 on smithi161 2024-06-01T01:54:26.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:26 smithi161 ceph-mon[26055]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-06-01T01:54:26.999 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:26 smithi116 ceph-mon[25082]: Deploying daemon osd.5 on smithi161 2024-06-01T01:54:26.999 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:26 smithi116 ceph-mon[25082]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-06-01T01:54:26.999 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:26 smithi116 ceph-mon[29958]: Deploying daemon osd.5 on smithi161 2024-06-01T01:54:26.999 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:26 smithi116 ceph-mon[29958]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-06-01T01:54:29.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:28 smithi116 ceph-mon[25082]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-06-01T01:54:29.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:28 smithi116 ceph-mon[29958]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-06-01T01:54:29.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:28 smithi161 ceph-mon[26055]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-06-01T01:54:30.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:30 smithi116 ceph-mon[25082]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-06-01T01:54:30.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:30 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:30.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:30 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:30.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:30 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:30.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:30 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:30.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:30 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:30.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:30 smithi116 ceph-mon[29958]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-06-01T01:54:30.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:30 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:30.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:30 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:30.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:30 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:30.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:30 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:30.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:30 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:30.318 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:30 smithi161 ceph-mon[26055]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-06-01T01:54:30.318 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:30 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:30.318 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:30 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:30.318 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:30 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:30.318 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:30 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:30.318 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:30 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:32.515 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 5 on host 'smithi161' 2024-06-01T01:54:33.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:32 smithi116 ceph-mon[25082]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-06-01T01:54:33.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:32 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:33.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:32 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:33.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:32 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:33.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:32 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:33.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:32 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:54:33.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:32 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:54:33.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:32 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:33.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:32 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:33.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:32 smithi116 ceph-mon[29958]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-06-01T01:54:33.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:32 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:33.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:32 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:33.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:32 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:33.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:32 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:33.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:32 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:54:33.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:32 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:54:33.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:32 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:33.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:32 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:33.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:32 smithi161 ceph-mon[26055]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-06-01T01:54:33.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:32 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:33.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:32 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:33.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:32 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:33.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:32 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:33.097 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:32 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:54:33.097 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:32 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:54:33.097 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:32 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:33.097 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:32 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:33.160 DEBUG:teuthology.orchestra.run.smithi161:osd.5> sudo journalctl -f -n 0 -u ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.5.service 2024-06-01T01:54:33.162 INFO:tasks.cephadm:Deploying osd.6 on smithi161 with /dev/vg_nvme/lv_2... 2024-06-01T01:54:33.163 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-01T01:54:33.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:33 smithi161 ceph-mon[26055]: Adjusting osd_memory_target on smithi161 to 8032M 2024-06-01T01:54:33.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:33 smithi161 ceph-mon[26055]: from='osd.5 [v2:172.21.15.161:6808/1993506828,v1:172.21.15.161:6809/1993506828]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-01T01:54:33.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:33 smithi161 ceph-mon[26055]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-01T01:54:34.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:33 smithi116 ceph-mon[25082]: Adjusting osd_memory_target on smithi161 to 8032M 2024-06-01T01:54:34.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:33 smithi116 ceph-mon[25082]: from='osd.5 [v2:172.21.15.161:6808/1993506828,v1:172.21.15.161:6809/1993506828]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-01T01:54:34.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:33 smithi116 ceph-mon[25082]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-01T01:54:34.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:33 smithi116 ceph-mon[29958]: Adjusting osd_memory_target on smithi161 to 8032M 2024-06-01T01:54:34.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:33 smithi116 ceph-mon[29958]: from='osd.5 [v2:172.21.15.161:6808/1993506828,v1:172.21.15.161:6809/1993506828]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-01T01:54:34.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:33 smithi116 ceph-mon[29958]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-01T01:54:35.033 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:34 smithi161 ceph-mon[26055]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-06-01T01:54:35.033 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:34 smithi161 ceph-mon[26055]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-01T01:54:35.033 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:34 smithi161 ceph-mon[26055]: osdmap e36: 6 total, 5 up, 6 in 2024-06-01T01:54:35.033 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:34 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T01:54:35.033 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:34 smithi161 ceph-mon[26055]: from='osd.5 [v2:172.21.15.161:6808/1993506828,v1:172.21.15.161:6809/1993506828]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T01:54:35.034 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:34 smithi161 ceph-mon[26055]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T01:54:35.034 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 01:54:34 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[33652]: 2024-06-01T01:54:34.761+0000 7f14304fc700 -1 osd.5 0 waiting for initial osdmap 2024-06-01T01:54:35.034 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 01:54:34 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[33652]: 2024-06-01T01:54:34.766+0000 7f142ce97700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-01T01:54:35.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:34 smithi116 ceph-mon[25082]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-06-01T01:54:35.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:34 smithi116 ceph-mon[25082]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-01T01:54:35.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:34 smithi116 ceph-mon[25082]: osdmap e36: 6 total, 5 up, 6 in 2024-06-01T01:54:35.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:34 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T01:54:35.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:34 smithi116 ceph-mon[25082]: from='osd.5 [v2:172.21.15.161:6808/1993506828,v1:172.21.15.161:6809/1993506828]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T01:54:35.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:34 smithi116 ceph-mon[25082]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T01:54:35.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:34 smithi116 ceph-mon[29958]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-06-01T01:54:35.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:34 smithi116 ceph-mon[29958]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-01T01:54:35.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:34 smithi116 ceph-mon[29958]: osdmap e36: 6 total, 5 up, 6 in 2024-06-01T01:54:35.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:34 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T01:54:35.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:34 smithi116 ceph-mon[29958]: from='osd.5 [v2:172.21.15.161:6808/1993506828,v1:172.21.15.161:6809/1993506828]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T01:54:35.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:34 smithi116 ceph-mon[29958]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T01:54:35.265 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-01T01:54:35.266 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-01T01:54:35.266 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-06-01T01:54:35.266 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-06-01T01:54:35.266 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0152323 s, 688 MB/s 2024-06-01T01:54:35.266 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-06-01T01:54:35.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:35 smithi161 ceph-mon[26055]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-06-01T01:54:35.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:35 smithi161 ceph-mon[26055]: osdmap e37: 6 total, 5 up, 6 in 2024-06-01T01:54:35.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:35 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T01:54:35.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:35 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T01:54:35.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:35 smithi161 ceph-mon[26055]: osd.5 [v2:172.21.15.161:6808/1993506828,v1:172.21.15.161:6809/1993506828] boot 2024-06-01T01:54:35.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:35 smithi161 ceph-mon[26055]: osdmap e38: 6 total, 6 up, 6 in 2024-06-01T01:54:35.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:35 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T01:54:35.883 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_2 2024-06-01T01:54:36.045 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:54:35 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:54:35] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:54:36.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:35 smithi116 ceph-mon[29958]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-06-01T01:54:36.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:35 smithi116 ceph-mon[29958]: osdmap e37: 6 total, 5 up, 6 in 2024-06-01T01:54:36.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:35 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T01:54:36.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:35 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T01:54:36.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:35 smithi116 ceph-mon[29958]: osd.5 [v2:172.21.15.161:6808/1993506828,v1:172.21.15.161:6809/1993506828] boot 2024-06-01T01:54:36.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:35 smithi116 ceph-mon[29958]: osdmap e38: 6 total, 6 up, 6 in 2024-06-01T01:54:36.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:35 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T01:54:36.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:35 smithi116 ceph-mon[25082]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-06-01T01:54:36.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:35 smithi116 ceph-mon[25082]: osdmap e37: 6 total, 5 up, 6 in 2024-06-01T01:54:36.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:35 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T01:54:36.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:35 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T01:54:36.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:35 smithi116 ceph-mon[25082]: osd.5 [v2:172.21.15.161:6808/1993506828,v1:172.21.15.161:6809/1993506828] boot 2024-06-01T01:54:36.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:35 smithi116 ceph-mon[25082]: osdmap e38: 6 total, 6 up, 6 in 2024-06-01T01:54:36.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:35 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T01:54:36.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:36 smithi161 ceph-mon[26055]: purged_snaps scrub starts 2024-06-01T01:54:36.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:36 smithi161 ceph-mon[26055]: purged_snaps scrub ok 2024-06-01T01:54:36.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:36 smithi161 ceph-mon[26055]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-06-01T01:54:36.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:36 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:36.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:36 smithi161 ceph-mon[26055]: Detected new or changed devices on smithi161 2024-06-01T01:54:36.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:36 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:36.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:36 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:36.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:36 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:36.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:36 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:36.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:36 smithi161 ceph-mon[26055]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:36.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:36 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:36.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:36 smithi161 ceph-mon[26055]: osdmap e39: 6 total, 6 up, 6 in 2024-06-01T01:54:37.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:36 smithi116 ceph-mon[25082]: purged_snaps scrub starts 2024-06-01T01:54:37.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:36 smithi116 ceph-mon[25082]: purged_snaps scrub ok 2024-06-01T01:54:37.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:36 smithi116 ceph-mon[25082]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-06-01T01:54:37.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:36 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:37.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:36 smithi116 ceph-mon[25082]: Detected new or changed devices on smithi161 2024-06-01T01:54:37.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:36 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:37.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:36 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:37.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:36 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:37.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:36 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:37.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:36 smithi116 ceph-mon[25082]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:37.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:36 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:37.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:36 smithi116 ceph-mon[25082]: osdmap e39: 6 total, 6 up, 6 in 2024-06-01T01:54:37.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:36 smithi116 ceph-mon[29958]: purged_snaps scrub starts 2024-06-01T01:54:37.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:36 smithi116 ceph-mon[29958]: purged_snaps scrub ok 2024-06-01T01:54:37.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:36 smithi116 ceph-mon[29958]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-06-01T01:54:37.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:36 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:37.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:36 smithi116 ceph-mon[29958]: Detected new or changed devices on smithi161 2024-06-01T01:54:37.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:36 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:37.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:36 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:37.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:36 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:37.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:36 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:37.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:36 smithi116 ceph-mon[29958]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:37.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:36 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:37.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:36 smithi116 ceph-mon[29958]: osdmap e39: 6 total, 6 up, 6 in 2024-06-01T01:54:37.921 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:37 smithi161 ceph-mon[26055]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-06-01T01:54:37.921 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:37 smithi161 ceph-mon[26055]: osdmap e40: 6 total, 6 up, 6 in 2024-06-01T01:54:38.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:37 smithi116 ceph-mon[25082]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-06-01T01:54:38.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:37 smithi116 ceph-mon[25082]: osdmap e40: 6 total, 6 up, 6 in 2024-06-01T01:54:38.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:37 smithi116 ceph-mon[29958]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-06-01T01:54:38.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:37 smithi116 ceph-mon[29958]: osdmap e40: 6 total, 6 up, 6 in 2024-06-01T01:54:38.545 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:54:38 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:54:38] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:54:39.024 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:38 smithi161 ceph-mon[26055]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:39.025 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:38 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:39.025 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:38 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:39.025 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:38 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:39.025 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:38 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:39.025 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:38 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:39.025 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:38 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:39.025 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:38 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-01T01:54:39.025 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:38 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-01T01:54:39.025 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:38 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:39.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:38 smithi116 ceph-mon[25082]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:39.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:38 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:39.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:38 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:39.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:38 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:39.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:38 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:39.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:38 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:39.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:38 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:39.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:38 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-01T01:54:39.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:38 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-01T01:54:39.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:38 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:39.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:38 smithi116 ceph-mon[29958]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:39.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:38 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:39.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:38 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:39.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:38 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:39.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:38 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:39.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:38 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:39.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:38 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:39.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:38 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-01T01:54:39.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:38 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-01T01:54:39.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:38 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:39.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:39 smithi161 ceph-mon[26055]: pgmap v136: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:39.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:39 smithi161 ceph-mon[26055]: from='client.24251 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:54:39.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:39 smithi161 ceph-mon[26055]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-06-01T01:54:39.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:39 smithi161 ceph-mon[26055]: Cluster is now healthy 2024-06-01T01:54:39.850 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:39 smithi161 ceph-mon[26055]: osdmap e41: 6 total, 6 up, 6 in 2024-06-01T01:54:40.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:39 smithi116 ceph-mon[29958]: pgmap v136: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:40.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:39 smithi116 ceph-mon[29958]: from='client.24251 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:54:40.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:39 smithi116 ceph-mon[29958]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-06-01T01:54:40.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:39 smithi116 ceph-mon[29958]: Cluster is now healthy 2024-06-01T01:54:40.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:39 smithi116 ceph-mon[29958]: osdmap e41: 6 total, 6 up, 6 in 2024-06-01T01:54:40.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:39 smithi116 ceph-mon[25082]: pgmap v136: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:40.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:39 smithi116 ceph-mon[25082]: from='client.24251 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:54:40.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:39 smithi116 ceph-mon[25082]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-06-01T01:54:40.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:39 smithi116 ceph-mon[25082]: Cluster is now healthy 2024-06-01T01:54:40.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:39 smithi116 ceph-mon[25082]: osdmap e41: 6 total, 6 up, 6 in 2024-06-01T01:54:41.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:40 smithi116 ceph-mon[25082]: osdmap e42: 6 total, 6 up, 6 in 2024-06-01T01:54:41.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:40 smithi116 ceph-mon[29958]: osdmap e42: 6 total, 6 up, 6 in 2024-06-01T01:54:41.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:40 smithi161 ceph-mon[26055]: osdmap e42: 6 total, 6 up, 6 in 2024-06-01T01:54:42.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:41 smithi116 ceph-mon[25082]: pgmap v139: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:42.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:41 smithi116 ceph-mon[29958]: pgmap v139: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:42.123 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:41 smithi161 ceph-mon[26055]: pgmap v139: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:43.045 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:54:42 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:54:42] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:54:44.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:43 smithi116 ceph-mon[29958]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:44.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:43 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:44.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:43 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:44.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:43 smithi116 ceph-mon[29958]: from='client.? 172.21.15.161:0/3524757239' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eefb882a-3b0c-4390-8875-625ef74c717a"}]: dispatch 2024-06-01T01:54:44.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:43 smithi116 ceph-mon[29958]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eefb882a-3b0c-4390-8875-625ef74c717a"}]: dispatch 2024-06-01T01:54:44.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:43 smithi116 ceph-mon[29958]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eefb882a-3b0c-4390-8875-625ef74c717a"}]': finished 2024-06-01T01:54:44.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:43 smithi116 ceph-mon[29958]: osdmap e43: 7 total, 6 up, 7 in 2024-06-01T01:54:44.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:43 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T01:54:44.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:43 smithi116 ceph-mon[29958]: from='client.? 172.21.15.161:0/572115339' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-01T01:54:44.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:43 smithi116 ceph-mon[25082]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:44.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:43 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:44.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:43 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:44.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:43 smithi116 ceph-mon[25082]: from='client.? 172.21.15.161:0/3524757239' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eefb882a-3b0c-4390-8875-625ef74c717a"}]: dispatch 2024-06-01T01:54:44.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:43 smithi116 ceph-mon[25082]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eefb882a-3b0c-4390-8875-625ef74c717a"}]: dispatch 2024-06-01T01:54:44.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:43 smithi116 ceph-mon[25082]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eefb882a-3b0c-4390-8875-625ef74c717a"}]': finished 2024-06-01T01:54:44.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:43 smithi116 ceph-mon[25082]: osdmap e43: 7 total, 6 up, 7 in 2024-06-01T01:54:44.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:43 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T01:54:44.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:43 smithi116 ceph-mon[25082]: from='client.? 172.21.15.161:0/572115339' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-01T01:54:44.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:43 smithi161 ceph-mon[26055]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:44.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:43 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:44.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:43 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:44.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:43 smithi161 ceph-mon[26055]: from='client.? 172.21.15.161:0/3524757239' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eefb882a-3b0c-4390-8875-625ef74c717a"}]: dispatch 2024-06-01T01:54:44.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:43 smithi161 ceph-mon[26055]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eefb882a-3b0c-4390-8875-625ef74c717a"}]: dispatch 2024-06-01T01:54:44.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:43 smithi161 ceph-mon[26055]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eefb882a-3b0c-4390-8875-625ef74c717a"}]': finished 2024-06-01T01:54:44.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:43 smithi161 ceph-mon[26055]: osdmap e43: 7 total, 6 up, 7 in 2024-06-01T01:54:44.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:43 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T01:54:44.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:43 smithi161 ceph-mon[26055]: from='client.? 172.21.15.161:0/572115339' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-01T01:54:45.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:45 smithi116 ceph-mon[25082]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:45.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:45 smithi116 ceph-mon[29958]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:45.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:45 smithi161 ceph-mon[26055]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:47.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:47 smithi161 ceph-mon[26055]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:47.477 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:47 smithi116 ceph-mon[29958]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:47.477 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:47 smithi116 ceph-mon[25082]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:49.626 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:49 smithi161 ceph-mon[26055]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:49.626 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:49 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-01T01:54:49.626 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:49 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:49.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:49 smithi116 ceph-mon[25082]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:49.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:49 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-01T01:54:49.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:49 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:49.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:49 smithi116 ceph-mon[29958]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:49.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:49 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-01T01:54:49.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:49 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:50.690 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:50 smithi161 ceph-mon[26055]: Deploying daemon osd.6 on smithi161 2024-06-01T01:54:50.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:50 smithi116 ceph-mon[25082]: Deploying daemon osd.6 on smithi161 2024-06-01T01:54:50.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:50 smithi116 ceph-mon[29958]: Deploying daemon osd.6 on smithi161 2024-06-01T01:54:51.690 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:51 smithi161 ceph-mon[26055]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:51.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:51 smithi116 ceph-mon[25082]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:51.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:51 smithi116 ceph-mon[29958]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:53.664 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:53 smithi161 ceph-mon[26055]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:53.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:53 smithi116 ceph-mon[25082]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:53.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:53 smithi116 ceph-mon[29958]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:55.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:55 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:55.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:55 smithi116 ceph-mon[29958]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:55.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:55 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:55.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:55 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:55.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:55 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:55.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:55 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:55.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:55 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:55.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:55 smithi116 ceph-mon[25082]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:55.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:55 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:55.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:55 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:55.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:55 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:55.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:55 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:55.631 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:55 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:55.632 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:55 smithi161 ceph-mon[26055]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:55.632 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:55 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:55.632 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:55 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:55.632 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:55 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:55.632 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:55 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:57.046 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 6 on host 'smithi161' 2024-06-01T01:54:57.646 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:57 smithi161 ceph-mon[26055]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:57.646 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:57 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:57.646 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:57 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:57.646 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:57 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:57.646 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:57 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:57.646 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:57 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:54:57.646 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:57 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:54:57.646 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:57 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:54:57.646 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:57 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:57.646 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:57 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:57.647 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:57 smithi161 ceph-mon[26055]: from='osd.6 [v2:172.21.15.161:6816/3451887914,v1:172.21.15.161:6817/3451887914]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-01T01:54:57.647 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:57 smithi161 ceph-mon[26055]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-01T01:54:57.669 DEBUG:teuthology.orchestra.run.smithi161:osd.6> sudo journalctl -f -n 0 -u ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.6.service 2024-06-01T01:54:57.674 INFO:tasks.cephadm:Deploying osd.7 on smithi161 with /dev/vg_nvme/lv_1... 2024-06-01T01:54:57.674 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-01T01:54:57.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:57 smithi116 ceph-mon[29958]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:57.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:57 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:57.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:57 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:57.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:57 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:57.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:57 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:57.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:57 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:54:57.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:57 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:54:57.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:57 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:54:57.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:57 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:57.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:57 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:57.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:57 smithi116 ceph-mon[29958]: from='osd.6 [v2:172.21.15.161:6816/3451887914,v1:172.21.15.161:6817/3451887914]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-01T01:54:57.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:57 smithi116 ceph-mon[29958]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-01T01:54:57.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:57 smithi116 ceph-mon[25082]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:57.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:57 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:57.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:57 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:57.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:57 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:57.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:57 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:57.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:57 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:54:57.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:57 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:54:57.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:57 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:54:57.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:57 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:57.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:57 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:57.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:57 smithi116 ceph-mon[25082]: from='osd.6 [v2:172.21.15.161:6816/3451887914,v1:172.21.15.161:6817/3451887914]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-01T01:54:57.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:57 smithi116 ceph-mon[25082]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-01T01:54:58.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:58 smithi161 ceph-mon[26055]: Adjusting osd_memory_target on smithi161 to 5354M 2024-06-01T01:54:58.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:58 smithi161 ceph-mon[26055]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-01T01:54:58.613 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:58 smithi161 ceph-mon[26055]: osdmap e44: 7 total, 6 up, 7 in 2024-06-01T01:54:58.613 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:58 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T01:54:58.613 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:58 smithi161 ceph-mon[26055]: from='osd.6 [v2:172.21.15.161:6816/3451887914,v1:172.21.15.161:6817/3451887914]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T01:54:58.613 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:58 smithi161 ceph-mon[26055]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T01:54:58.613 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:58 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:58.613 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:58 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:58.795 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:54:58 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:54:58] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:54:58.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:58 smithi116 ceph-mon[29958]: Adjusting osd_memory_target on smithi161 to 5354M 2024-06-01T01:54:58.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:58 smithi116 ceph-mon[29958]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-01T01:54:58.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:58 smithi116 ceph-mon[29958]: osdmap e44: 7 total, 6 up, 7 in 2024-06-01T01:54:58.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:58 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T01:54:58.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:58 smithi116 ceph-mon[29958]: from='osd.6 [v2:172.21.15.161:6816/3451887914,v1:172.21.15.161:6817/3451887914]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T01:54:58.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:58 smithi116 ceph-mon[29958]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T01:54:58.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:58 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:58.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:58 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:58.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:58 smithi116 ceph-mon[25082]: Adjusting osd_memory_target on smithi161 to 5354M 2024-06-01T01:54:58.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:58 smithi116 ceph-mon[25082]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-01T01:54:58.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:58 smithi116 ceph-mon[25082]: osdmap e44: 7 total, 6 up, 7 in 2024-06-01T01:54:58.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:58 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T01:54:58.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:58 smithi116 ceph-mon[25082]: from='osd.6 [v2:172.21.15.161:6816/3451887914,v1:172.21.15.161:6817/3451887914]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T01:54:58.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:58 smithi116 ceph-mon[25082]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T01:54:58.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:58 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:58.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:58 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:59.197 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 01:54:59 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[36538]: 2024-06-01T01:54:59.117+0000 7f63d7803700 -1 osd.6 0 waiting for initial osdmap 2024-06-01T01:54:59.197 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 01:54:59 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[36538]: 2024-06-01T01:54:59.124+0000 7f63d0997700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-01T01:54:59.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:59 smithi161 ceph-mon[26055]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:59.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:59 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:59.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:59 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:59.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:59 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:59.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:59 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:59.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:59 smithi161 ceph-mon[26055]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-06-01T01:54:59.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:59 smithi161 ceph-mon[26055]: osdmap e45: 7 total, 6 up, 7 in 2024-06-01T01:54:59.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:59 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T01:54:59.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:54:59 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T01:54:59.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:59 smithi116 ceph-mon[29958]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:59.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:59 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:59.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:59 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:59.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:59 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:59.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:59 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:59.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:59 smithi116 ceph-mon[29958]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-06-01T01:54:59.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:59 smithi116 ceph-mon[29958]: osdmap e45: 7 total, 6 up, 7 in 2024-06-01T01:54:59.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:59 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T01:54:59.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:54:59 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T01:54:59.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:59 smithi116 ceph-mon[25082]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-06-01T01:54:59.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:59 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:54:59.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:59 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:54:59.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:59 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:54:59.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:59 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:54:59.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:59 smithi116 ceph-mon[25082]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-06-01T01:54:59.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:59 smithi116 ceph-mon[25082]: osdmap e45: 7 total, 6 up, 7 in 2024-06-01T01:54:59.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:59 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T01:54:59.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:54:59 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T01:55:01.433 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:01 smithi161 ceph-mon[26055]: purged_snaps scrub starts 2024-06-01T01:55:01.433 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:01 smithi161 ceph-mon[26055]: purged_snaps scrub ok 2024-06-01T01:55:01.434 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:01 smithi161 ceph-mon[26055]: osd.6 [v2:172.21.15.161:6816/3451887914,v1:172.21.15.161:6817/3451887914] boot 2024-06-01T01:55:01.434 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:01 smithi161 ceph-mon[26055]: osdmap e46: 7 total, 7 up, 7 in 2024-06-01T01:55:01.434 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:01 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T01:55:01.434 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:01 smithi161 ceph-mon[26055]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2024-06-01T01:55:01.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:01 smithi116 ceph-mon[25082]: purged_snaps scrub starts 2024-06-01T01:55:01.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:01 smithi116 ceph-mon[25082]: purged_snaps scrub ok 2024-06-01T01:55:01.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:01 smithi116 ceph-mon[25082]: osd.6 [v2:172.21.15.161:6816/3451887914,v1:172.21.15.161:6817/3451887914] boot 2024-06-01T01:55:01.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:01 smithi116 ceph-mon[25082]: osdmap e46: 7 total, 7 up, 7 in 2024-06-01T01:55:01.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:01 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T01:55:01.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:01 smithi116 ceph-mon[25082]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2024-06-01T01:55:01.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:01 smithi116 ceph-mon[29958]: purged_snaps scrub starts 2024-06-01T01:55:01.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:01 smithi116 ceph-mon[29958]: purged_snaps scrub ok 2024-06-01T01:55:01.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:01 smithi116 ceph-mon[29958]: osd.6 [v2:172.21.15.161:6816/3451887914,v1:172.21.15.161:6817/3451887914] boot 2024-06-01T01:55:01.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:01 smithi116 ceph-mon[29958]: osdmap e46: 7 total, 7 up, 7 in 2024-06-01T01:55:01.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:01 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T01:55:01.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:01 smithi116 ceph-mon[29958]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2024-06-01T01:55:02.113 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:01 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:55:01] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:55:02.217 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-01T01:55:02.218 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-01T01:55:02.218 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-06-01T01:55:02.218 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-06-01T01:55:02.218 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123909 s, 846 MB/s 2024-06-01T01:55:02.218 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-06-01T01:55:02.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:02 smithi161 ceph-mon[26055]: osdmap e47: 7 total, 7 up, 7 in 2024-06-01T01:55:02.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:02 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:02.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:02 smithi161 ceph-mon[26055]: Detected new or changed devices on smithi161 2024-06-01T01:55:02.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:02 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:02.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:02 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:02.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:02 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:02.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:02 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:55:02.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:02 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:02.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:02 smithi116 ceph-mon[25082]: osdmap e47: 7 total, 7 up, 7 in 2024-06-01T01:55:02.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:02 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:02.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:02 smithi116 ceph-mon[25082]: Detected new or changed devices on smithi161 2024-06-01T01:55:02.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:02 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:02.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:02 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:02.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:02 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:02.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:02 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:55:02.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:02 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:02.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:02 smithi116 ceph-mon[29958]: osdmap e47: 7 total, 7 up, 7 in 2024-06-01T01:55:02.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:02 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:02.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:02 smithi116 ceph-mon[29958]: Detected new or changed devices on smithi161 2024-06-01T01:55:02.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:02 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:02.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:02 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:02.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:02 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:02.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:02 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:55:02.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:02 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:02.889 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_1 2024-06-01T01:55:03.124 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:55:02] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:55:03.432 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:03 smithi161 ceph-mon[26055]: osdmap e48: 7 total, 7 up, 7 in 2024-06-01T01:55:03.432 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:03 smithi161 ceph-mon[26055]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2024-06-01T01:55:03.432 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:03 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:03.432 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:03 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:03.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:03 smithi116 ceph-mon[25082]: osdmap e48: 7 total, 7 up, 7 in 2024-06-01T01:55:03.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:03 smithi116 ceph-mon[25082]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2024-06-01T01:55:03.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:03 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:03.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:03 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:03.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:03 smithi116 ceph-mon[29958]: osdmap e48: 7 total, 7 up, 7 in 2024-06-01T01:55:03.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:03 smithi116 ceph-mon[29958]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2024-06-01T01:55:03.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:03 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:03.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:03 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:04.407 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:04 smithi161 ceph-mon[26055]: osdmap e49: 7 total, 7 up, 7 in 2024-06-01T01:55:04.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:04 smithi116 ceph-mon[25082]: osdmap e49: 7 total, 7 up, 7 in 2024-06-01T01:55:04.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:04 smithi116 ceph-mon[29958]: osdmap e49: 7 total, 7 up, 7 in 2024-06-01T01:55:05.400 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:05 smithi161 ceph-mon[26055]: osdmap e50: 7 total, 7 up, 7 in 2024-06-01T01:55:05.400 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:05 smithi161 ceph-mon[26055]: pgmap v159: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-06-01T01:55:05.400 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:05 smithi161 ceph-mon[26055]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:55:05.400 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:05 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-01T01:55:05.400 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:05 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-01T01:55:05.400 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:05 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:05.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:05 smithi116 ceph-mon[25082]: osdmap e50: 7 total, 7 up, 7 in 2024-06-01T01:55:05.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:05 smithi116 ceph-mon[25082]: pgmap v159: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-06-01T01:55:05.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:05 smithi116 ceph-mon[25082]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:55:05.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:05 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-01T01:55:05.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:05 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-01T01:55:05.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:05 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:05.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:05 smithi116 ceph-mon[29958]: osdmap e50: 7 total, 7 up, 7 in 2024-06-01T01:55:05.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:05 smithi116 ceph-mon[29958]: pgmap v159: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-06-01T01:55:05.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:05 smithi116 ceph-mon[29958]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:55:05.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:05 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-01T01:55:05.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:05 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-01T01:55:05.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:05 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:07.655 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:07 smithi116 ceph-mon[29958]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-06-01T01:55:07.656 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:07 smithi116 ceph-mon[29958]: from='client.? 172.21.15.161:0/1052369031' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "704f8550-799e-4d55-9e02-c5ec18173973"}]: dispatch 2024-06-01T01:55:07.656 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:07 smithi116 ceph-mon[29958]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "704f8550-799e-4d55-9e02-c5ec18173973"}]: dispatch 2024-06-01T01:55:07.656 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:07 smithi116 ceph-mon[29958]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "704f8550-799e-4d55-9e02-c5ec18173973"}]': finished 2024-06-01T01:55:07.656 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:07 smithi116 ceph-mon[29958]: osdmap e51: 8 total, 7 up, 8 in 2024-06-01T01:55:07.656 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:07 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T01:55:07.656 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:07 smithi116 ceph-mon[25082]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-06-01T01:55:07.656 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:07 smithi116 ceph-mon[25082]: from='client.? 172.21.15.161:0/1052369031' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "704f8550-799e-4d55-9e02-c5ec18173973"}]: dispatch 2024-06-01T01:55:07.656 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:07 smithi116 ceph-mon[25082]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "704f8550-799e-4d55-9e02-c5ec18173973"}]: dispatch 2024-06-01T01:55:07.656 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:07 smithi116 ceph-mon[25082]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "704f8550-799e-4d55-9e02-c5ec18173973"}]': finished 2024-06-01T01:55:07.656 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:07 smithi116 ceph-mon[25082]: osdmap e51: 8 total, 7 up, 8 in 2024-06-01T01:55:07.657 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:07 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T01:55:07.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:07 smithi161 ceph-mon[26055]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-06-01T01:55:07.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:07 smithi161 ceph-mon[26055]: from='client.? 172.21.15.161:0/1052369031' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "704f8550-799e-4d55-9e02-c5ec18173973"}]: dispatch 2024-06-01T01:55:07.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:07 smithi161 ceph-mon[26055]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "704f8550-799e-4d55-9e02-c5ec18173973"}]: dispatch 2024-06-01T01:55:07.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:07 smithi161 ceph-mon[26055]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "704f8550-799e-4d55-9e02-c5ec18173973"}]': finished 2024-06-01T01:55:07.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:07 smithi161 ceph-mon[26055]: osdmap e51: 8 total, 7 up, 8 in 2024-06-01T01:55:07.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:07 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T01:55:08.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:08 smithi161 ceph-mon[26055]: from='client.? 172.21.15.161:0/1964646818' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-01T01:55:08.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:08 smithi116 ceph-mon[25082]: from='client.? 172.21.15.161:0/1964646818' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-01T01:55:08.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:08 smithi116 ceph-mon[29958]: from='client.? 172.21.15.161:0/1964646818' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-01T01:55:09.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:09 smithi161 ceph-mon[26055]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-06-01T01:55:09.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:09 smithi116 ceph-mon[25082]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-06-01T01:55:09.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:09 smithi116 ceph-mon[29958]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-06-01T01:55:11.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:11 smithi161 ceph-mon[26055]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-06-01T01:55:11.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:11 smithi116 ceph-mon[25082]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-06-01T01:55:11.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:11 smithi116 ceph-mon[29958]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-06-01T01:55:13.628 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:13 smithi161 ceph-mon[26055]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-06-01T01:55:13.629 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:13 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T01:55:13.629 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:13 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T01:55:13.629 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:13 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-01T01:55:13.629 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:13 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:13.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:13 smithi116 ceph-mon[25082]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-06-01T01:55:13.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:13 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T01:55:13.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:13 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T01:55:13.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:13 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-01T01:55:13.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:13 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:13.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:13 smithi116 ceph-mon[29958]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-06-01T01:55:13.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:13 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T01:55:13.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:13 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T01:55:13.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:13 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-01T01:55:13.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:13 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:14.506 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:14 smithi161 ceph-mon[26055]: Deploying daemon osd.7 on smithi161 2024-06-01T01:55:14.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:14 smithi116 ceph-mon[25082]: Deploying daemon osd.7 on smithi161 2024-06-01T01:55:14.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:14 smithi116 ceph-mon[29958]: Deploying daemon osd.7 on smithi161 2024-06-01T01:55:15.429 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:15 smithi161 ceph-mon[26055]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-06-01T01:55:15.696 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:15 smithi116 ceph-mon[25082]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-06-01T01:55:15.696 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:15 smithi116 ceph-mon[29958]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-06-01T01:55:17.643 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:17 smithi161 ceph-mon[26055]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-06-01T01:55:17.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:17 smithi116 ceph-mon[25082]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-06-01T01:55:17.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:17 smithi116 ceph-mon[29958]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-06-01T01:55:19.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:18 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:19.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:18 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:19.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:18 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:19.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:18 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:55:19.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:18 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:19.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:18 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:19.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:18 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:19.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:18 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:19.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:18 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:55:19.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:18 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:19.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:18 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:19.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:18 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:19.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:18 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:19.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:18 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:55:19.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:18 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:20.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:19 smithi161 ceph-mon[26055]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-06-01T01:55:20.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:19 smithi116 ceph-mon[25082]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-06-01T01:55:20.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:19 smithi116 ceph-mon[29958]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-06-01T01:55:22.041 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:21 smithi161 ceph-mon[26055]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-06-01T01:55:22.042 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:21 smithi161 ceph-mon[26055]: from='osd.7 [v2:172.21.15.161:6824/2386571128,v1:172.21.15.161:6825/2386571128]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-01T01:55:22.042 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:21 smithi161 ceph-mon[26055]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-01T01:55:22.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:21 smithi116 ceph-mon[25082]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-06-01T01:55:22.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:21 smithi116 ceph-mon[25082]: from='osd.7 [v2:172.21.15.161:6824/2386571128,v1:172.21.15.161:6825/2386571128]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-01T01:55:22.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:21 smithi116 ceph-mon[25082]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-01T01:55:22.296 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:21 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:55:21] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:55:22.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:21 smithi116 ceph-mon[29958]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-06-01T01:55:22.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:21 smithi116 ceph-mon[29958]: from='osd.7 [v2:172.21.15.161:6824/2386571128,v1:172.21.15.161:6825/2386571128]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-01T01:55:22.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:21 smithi116 ceph-mon[29958]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-01T01:55:22.760 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 7 on host 'smithi161' 2024-06-01T01:55:23.144 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:22 smithi161 ceph-mon[26055]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-01T01:55:23.145 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:22 smithi161 ceph-mon[26055]: osdmap e52: 8 total, 7 up, 8 in 2024-06-01T01:55:23.145 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:22 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T01:55:23.145 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:22 smithi161 ceph-mon[26055]: from='osd.7 [v2:172.21.15.161:6824/2386571128,v1:172.21.15.161:6825/2386571128]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T01:55:23.145 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:22 smithi161 ceph-mon[26055]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T01:55:23.145 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:22 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:23.146 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:22 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:23.146 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:22 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:23.146 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:22 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:23.146 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:22 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:55:23.146 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:22 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:23.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:22 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:23.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:22 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:23.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:22 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:23.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:22 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:55:23.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:22 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:23.295 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:55:22] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:55:23.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-mon[29958]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-01T01:55:23.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-mon[29958]: osdmap e52: 8 total, 7 up, 8 in 2024-06-01T01:55:23.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T01:55:23.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-mon[29958]: from='osd.7 [v2:172.21.15.161:6824/2386571128,v1:172.21.15.161:6825/2386571128]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T01:55:23.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-mon[29958]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T01:55:23.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:23.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:23.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:23.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:23.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:55:23.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:23.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:23.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:23.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:23.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:55:23.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:23.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-mon[25082]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-01T01:55:23.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-mon[25082]: osdmap e52: 8 total, 7 up, 8 in 2024-06-01T01:55:23.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T01:55:23.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-mon[25082]: from='osd.7 [v2:172.21.15.161:6824/2386571128,v1:172.21.15.161:6825/2386571128]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T01:55:23.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-mon[25082]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T01:55:23.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:23.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:23.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:23.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:23.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:55:23.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:23.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:23.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:23.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:23.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:55:23.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:22 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:23.412 DEBUG:teuthology.orchestra.run.smithi161:osd.7> sudo journalctl -f -n 0 -u ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.7.service 2024-06-01T01:55:23.415 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-06-01T01:55:23.415 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph osd stat -f json 2024-06-01T01:55:23.741 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:23 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:55:23] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:55:24.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:23 smithi116 ceph-mon[29958]: Detected new or changed devices on smithi161 2024-06-01T01:55:24.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:23 smithi116 ceph-mon[29958]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-06-01T01:55:24.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:23 smithi116 ceph-mon[29958]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-06-01T01:55:24.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:23 smithi116 ceph-mon[29958]: osdmap e53: 8 total, 7 up, 8 in 2024-06-01T01:55:24.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:23 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T01:55:24.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:23 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T01:55:24.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:23 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:24.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:23 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:24.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:23 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:24.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:23 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:24.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:23 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:24.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:23 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:24.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:23 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:55:24.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:23 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:24.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:23 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T01:55:24.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:23 smithi116 ceph-mon[25082]: Detected new or changed devices on smithi161 2024-06-01T01:55:24.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:23 smithi116 ceph-mon[25082]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-06-01T01:55:24.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:23 smithi116 ceph-mon[25082]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-06-01T01:55:24.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:23 smithi116 ceph-mon[25082]: osdmap e53: 8 total, 7 up, 8 in 2024-06-01T01:55:24.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:23 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T01:55:24.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:23 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T01:55:24.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:23 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:24.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:23 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:24.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:23 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:24.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:23 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:24.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:23 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:24.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:23 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:24.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:23 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:55:24.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:23 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:24.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:23 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T01:55:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:23 smithi161 ceph-mon[26055]: Detected new or changed devices on smithi161 2024-06-01T01:55:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:23 smithi161 ceph-mon[26055]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-06-01T01:55:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:23 smithi161 ceph-mon[26055]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-06-01T01:55:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:23 smithi161 ceph-mon[26055]: osdmap e53: 8 total, 7 up, 8 in 2024-06-01T01:55:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:23 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T01:55:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:23 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T01:55:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:23 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:23 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:23 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:24.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:23 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:24.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:23 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:24.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:23 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:24.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:23 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:55:24.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:23 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:24.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:23 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T01:55:25.079 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:24 smithi116 ceph-mon[25082]: purged_snaps scrub starts 2024-06-01T01:55:25.079 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:24 smithi116 ceph-mon[25082]: purged_snaps scrub ok 2024-06-01T01:55:25.079 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:24 smithi116 ceph-mon[25082]: osd.7 [v2:172.21.15.161:6824/2386571128,v1:172.21.15.161:6825/2386571128] boot 2024-06-01T01:55:25.079 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:24 smithi116 ceph-mon[25082]: osdmap e54: 8 total, 8 up, 8 in 2024-06-01T01:55:25.079 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:24 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T01:55:25.079 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:24 smithi116 ceph-mon[25082]: osdmap e55: 8 total, 8 up, 8 in 2024-06-01T01:55:25.079 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:24 smithi116 ceph-mon[25082]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-06-01T01:55:25.080 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:24 smithi116 ceph-mon[29958]: purged_snaps scrub starts 2024-06-01T01:55:25.080 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:24 smithi116 ceph-mon[29958]: purged_snaps scrub ok 2024-06-01T01:55:25.080 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:24 smithi116 ceph-mon[29958]: osd.7 [v2:172.21.15.161:6824/2386571128,v1:172.21.15.161:6825/2386571128] boot 2024-06-01T01:55:25.080 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:24 smithi116 ceph-mon[29958]: osdmap e54: 8 total, 8 up, 8 in 2024-06-01T01:55:25.080 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:24 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T01:55:25.080 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:24 smithi116 ceph-mon[29958]: osdmap e55: 8 total, 8 up, 8 in 2024-06-01T01:55:25.080 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:24 smithi116 ceph-mon[29958]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-06-01T01:55:25.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:24 smithi161 ceph-mon[26055]: purged_snaps scrub starts 2024-06-01T01:55:25.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:24 smithi161 ceph-mon[26055]: purged_snaps scrub ok 2024-06-01T01:55:25.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:24 smithi161 ceph-mon[26055]: osd.7 [v2:172.21.15.161:6824/2386571128,v1:172.21.15.161:6825/2386571128] boot 2024-06-01T01:55:25.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:24 smithi161 ceph-mon[26055]: osdmap e54: 8 total, 8 up, 8 in 2024-06-01T01:55:25.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:24 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T01:55:25.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:24 smithi161 ceph-mon[26055]: osdmap e55: 8 total, 8 up, 8 in 2024-06-01T01:55:25.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:24 smithi161 ceph-mon[26055]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-06-01T01:55:25.494 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-06-01T01:55:26.138 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":56,"num_osds":8,"num_up_osds":8,"osd_up_since":1717206923,"num_in_osds":8,"osd_in_since":1717206907,"num_remapped_pgs":1} 2024-06-01T01:55:26.139 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-01T01:55:26.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:26 smithi116 ceph-mon[29958]: osdmap e56: 8 total, 8 up, 8 in 2024-06-01T01:55:26.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:26 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1910460108' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-01T01:55:26.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:26 smithi116 ceph-mon[25082]: osdmap e56: 8 total, 8 up, 8 in 2024-06-01T01:55:26.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:26 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1910460108' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-01T01:55:26.348 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/mon.a/config 2024-06-01T01:55:26.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:26 smithi161 ceph-mon[26055]: osdmap e56: 8 total, 8 up, 8 in 2024-06-01T01:55:26.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:26 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1910460108' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-01T01:55:27.342 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:27 smithi116 ceph-mon[25082]: osdmap e57: 8 total, 8 up, 8 in 2024-06-01T01:55:27.342 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:27 smithi116 ceph-mon[25082]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:55:27.342 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:27 smithi116 ceph-mon[29958]: osdmap e57: 8 total, 8 up, 8 in 2024-06-01T01:55:27.342 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:27 smithi116 ceph-mon[29958]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:55:27.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:27 smithi161 ceph-mon[26055]: osdmap e57: 8 total, 8 up, 8 in 2024-06-01T01:55:27.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:27 smithi161 ceph-mon[26055]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:55:28.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:28 smithi116 ceph-mon[25082]: osdmap e58: 8 total, 8 up, 8 in 2024-06-01T01:55:28.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:28 smithi116 ceph-mon[29958]: osdmap e58: 8 total, 8 up, 8 in 2024-06-01T01:55:28.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:28 smithi161 ceph-mon[26055]: osdmap e58: 8 total, 8 up, 8 in 2024-06-01T01:55:29.135 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:29 smithi116 ceph-mon[25082]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:55:29.135 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:29 smithi116 ceph-mon[29958]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:55:29.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:29 smithi161 ceph-mon[26055]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:55:29.638 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-06-01T01:55:29.638 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":58,"fsid":"393f964a-1fb9-11ef-bc9b-c7b262605968","created":"2024-06-01T01:50:23.493545+0000","modified":"2024-06-01T01:55:27.064515+0000","last_up_change":"2024-06-01T01:55:23.915950+0000","last_in_change":"2024-06-01T01:55:07.058813+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-01T01:53:25.218001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"77d7478a-9bf5-473d-8296-b0c91060b912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":1460215654},{"type":"v1","addr":"172.21.15.116:6803","nonce":1460215654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":1460215654},{"type":"v1","addr":"172.21.15.116:6805","nonce":1460215654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":1460215654},{"type":"v1","addr":"172.21.15.116:6809","nonce":1460215654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":1460215654},{"type":"v1","addr":"172.21.15.116:6807","nonce":1460215654}]},"public_addr":"172.21.15.116:6803/1460215654","cluster_addr":"172.21.15.116:6805/1460215654","heartbeat_back_addr":"172.21.15.116:6809/1460215654","heartbeat_front_addr":"172.21.15.116:6807/1460215654","state":["exists","up"]},{"osd":1,"uuid":"95343a32-622a-4716-8fcc-71ce91004700","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":166790657},{"type":"v1","addr":"172.21.15.116:6811","nonce":166790657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":166790657},{"type":"v1","addr":"172.21.15.116:6813","nonce":166790657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":166790657},{"type":"v1","addr":"172.21.15.116:6817","nonce":166790657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":166790657},{"type":"v1","addr":"172.21.15.116:6815","nonce":166790657}]},"public_addr":"172.21.15.116:6811/166790657","cluster_addr":"172.21.15.116:6813/166790657","heartbeat_back_addr":"172.21.15.116:6817/166790657","heartbeat_front_addr":"172.21.15.116:6815/166790657","state":["exists","up"]},{"osd":2,"uuid":"d392c3f3-f08e-48d0-aa79-727eb1f9c71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2956549862},{"type":"v1","addr":"172.21.15.116:6819","nonce":2956549862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2956549862},{"type":"v1","addr":"172.21.15.116:6821","nonce":2956549862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2956549862},{"type":"v1","addr":"172.21.15.116:6825","nonce":2956549862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2956549862},{"type":"v1","addr":"172.21.15.116:6823","nonce":2956549862}]},"public_addr":"172.21.15.116:6819/2956549862","cluster_addr":"172.21.15.116:6821/2956549862","heartbeat_back_addr":"172.21.15.116:6825/2956549862","heartbeat_front_addr":"172.21.15.116:6823/2956549862","state":["exists","up"]},{"osd":3,"uuid":"73a18daf-1e3e-476c-8375-96a67d726ef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3228322598},{"type":"v1","addr":"172.21.15.116:6827","nonce":3228322598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3228322598},{"type":"v1","addr":"172.21.15.116:6829","nonce":3228322598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3228322598},{"type":"v1","addr":"172.21.15.116:6833","nonce":3228322598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3228322598},{"type":"v1","addr":"172.21.15.116:6831","nonce":3228322598}]},"public_addr":"172.21.15.116:6827/3228322598","cluster_addr":"172.21.15.116:6829/3228322598","heartbeat_back_addr":"172.21.15.116:6833/3228322598","heartbeat_front_addr":"172.21.15.116:6831/3228322598","state":["exists","up"]},{"osd":4,"uuid":"6b1b3903-2ee8-4483-aeac-e6ffb3eff7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":147846900},{"type":"v1","addr":"172.21.15.161:6801","nonce":147846900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":147846900},{"type":"v1","addr":"172.21.15.161:6803","nonce":147846900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":147846900},{"type":"v1","addr":"172.21.15.161:6807","nonce":147846900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":147846900},{"type":"v1","addr":"172.21.15.161:6805","nonce":147846900}]},"public_addr":"172.21.15.161:6801/147846900","cluster_addr":"172.21.15.161:6803/147846900","heartbeat_back_addr":"172.21.15.161:6807/147846900","heartbeat_front_addr":"172.21.15.161:6805/147846900","state":["exists","up"]},{"osd":5,"uuid":"8f14db41-92df-409b-9fbd-070185f04e0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1993506828},{"type":"v1","addr":"172.21.15.161:6809","nonce":1993506828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1993506828},{"type":"v1","addr":"172.21.15.161:6811","nonce":1993506828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1993506828},{"type":"v1","addr":"172.21.15.161:6815","nonce":1993506828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1993506828},{"type":"v1","addr":"172.21.15.161:6813","nonce":1993506828}]},"public_addr":"172.21.15.161:6809/1993506828","cluster_addr":"172.21.15.161:6811/1993506828","heartbeat_back_addr":"172.21.15.161:6815/1993506828","heartbeat_front_addr":"172.21.15.161:6813/1993506828","state":["exists","up"]},{"osd":6,"uuid":"eefb882a-3b0c-4390-8875-625ef74c717a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3451887914},{"type":"v1","addr":"172.21.15.161:6817","nonce":3451887914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":3451887914},{"type":"v1","addr":"172.21.15.161:6819","nonce":3451887914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":3451887914},{"type":"v1","addr":"172.21.15.161:6823","nonce":3451887914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":3451887914},{"type":"v1","addr":"172.21.15.161:6821","nonce":3451887914}]},"public_addr":"172.21.15.161:6817/3451887914","cluster_addr":"172.21.15.161:6819/3451887914","heartbeat_back_addr":"172.21.15.161:6823/3451887914","heartbeat_front_addr":"172.21.15.161:6821/3451887914","state":["exists","up"]},{"osd":7,"uuid":"704f8550-799e-4d55-9e02-c5ec18173973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2386571128},{"type":"v1","addr":"172.21.15.161:6825","nonce":2386571128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2386571128},{"type":"v1","addr":"172.21.15.161:6827","nonce":2386571128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2386571128},{"type":"v1","addr":"172.21.15.161:6831","nonce":2386571128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2386571128},{"type":"v1","addr":"172.21.15.161:6829","nonce":2386571128}]},"public_addr":"172.21.15.161:6825/2386571128","cluster_addr":"172.21.15.161:6827/2386571128","heartbeat_back_addr":"172.21.15.161:6831/2386571128","heartbeat_front_addr":"172.21.15.161:6829/2386571128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-06-01T01:52:34.917939+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-06-01T01:52:58.344980+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-06-01T01:53:22.397763+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-06-01T01:53:47.760963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-06-01T01:54:11.293430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-06-01T01:54:34.125895+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-06-01T01:54:58.321697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-06-01T01:55:22.013822+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.116:0/3769678136":"2024-06-02T01:51:12.859459+0000","172.21.15.116:6801/3557119511":"2024-06-02T01:51:12.859459+0000","172.21.15.116:6800/3557119511":"2024-06-02T01:51:12.859459+0000","172.21.15.116:0/2080484873":"2024-06-02T01:51:12.859459+0000","172.21.15.116:0/1985344094":"2024-06-02T01:51:12.859459+0000","172.21.15.116:0/455415098":"2024-06-02T01:50:49.481521+0000","172.21.15.116:0/3329906888":"2024-06-02T01:50:49.481521+0000","172.21.15.116:0/3120590326":"2024-06-02T01:50:49.481521+0000","172.21.15.116:6800/2138133985":"2024-06-02T01:50:49.481521+0000","172.21.15.116:6801/2138133985":"2024-06-02T01:50:49.481521+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-01T01:55:30.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:30 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/2382109601' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-01T01:55:30.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:30 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/2382109601' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-01T01:55:30.412 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-01T01:53:25.218001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-06-01T01:55:30.412 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-06-01T01:55:30.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:30 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/2382109601' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-01T01:55:31.158 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/mon.a/config 2024-06-01T01:55:31.388 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:31 smithi116 ceph-mon[29958]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-06-01T01:55:31.389 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:31 smithi116 ceph-mon[25082]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-06-01T01:55:31.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:31 smithi161 ceph-mon[26055]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-06-01T01:55:33.000 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2024-06-01T01:55:33.638 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:33 smithi116 ceph-mon[25082]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-06-01T01:55:33.639 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:33 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/2699843037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-01T01:55:33.639 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:33 smithi116 ceph-mon[29958]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-06-01T01:55:33.639 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:33 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/2699843037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-01T01:55:33.665 INFO:tasks.cephadm:Adding prometheus.a on smithi161 2024-06-01T01:55:33.666 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph orch apply prometheus '1;smithi161=a' 2024-06-01T01:55:33.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:33 smithi161 ceph-mon[26055]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-06-01T01:55:33.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:33 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/2699843037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-01T01:55:35.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:35 smithi161 ceph-mon[26055]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-06-01T01:55:35.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:35 smithi116 ceph-mon[25082]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-06-01T01:55:35.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:35 smithi116 ceph-mon[29958]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-06-01T01:55:35.601 INFO:teuthology.orchestra.run.smithi161.stdout:Scheduled prometheus update... 2024-06-01T01:55:36.201 DEBUG:teuthology.orchestra.run.smithi161:prometheus.a> sudo journalctl -f -n 0 -u ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@prometheus.a.service 2024-06-01T01:55:36.245 INFO:tasks.cephadm:Adding node-exporter.a on smithi116 2024-06-01T01:55:36.245 INFO:tasks.cephadm:Adding node-exporter.b on smithi161 2024-06-01T01:55:36.245 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph orch apply node-exporter '2;smithi116=a;smithi161=b' 2024-06-01T01:55:36.914 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:36 smithi161 ceph-mon[26055]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi161=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:55:36.914 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:36 smithi161 ceph-mon[26055]: Saving service prometheus spec with placement smithi161=a;count:1 2024-06-01T01:55:36.914 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:36 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:36.914 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:36 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:36.914 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:36 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:36.914 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:36 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:55:36.914 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:36 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:36.914 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:36 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-06-01T01:55:36.915 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:36 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: ignoring --setuser ceph since I am not root 2024-06-01T01:55:36.915 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:36 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: ignoring --setgroup ceph since I am not root 2024-06-01T01:55:36.915 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:36 smithi161 ceph-mgr[27604]: -- 172.21.15.161:0/1071564370 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5576054b2340 con 0x55760548ac00 2024-06-01T01:55:36.915 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:36 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:55:36.823+0000 7f650c723000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-01T01:55:36.954 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:36 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ignoring --setuser ceph since I am not root 2024-06-01T01:55:36.954 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:36 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ignoring --setgroup ceph since I am not root 2024-06-01T01:55:36.955 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:36 smithi116 ceph-mgr[25316]: -- 172.21.15.116:0/91807172 <== mon.1 v2:172.21.15.116:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561c34684340 con 0x561c3465cc00 2024-06-01T01:55:36.955 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:36 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:55:36.830+0000 7ff5ce0b0000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-01T01:55:36.955 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:36 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:55:36.956+0000 7ff5ce0b0000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-01T01:55:36.955 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:36 smithi116 ceph-mon[25082]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi161=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:55:36.955 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:36 smithi116 ceph-mon[25082]: Saving service prometheus spec with placement smithi161=a;count:1 2024-06-01T01:55:36.955 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:36 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:36.955 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:36 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:36.955 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:36 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:36.956 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:36 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:55:36.956 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:36 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:36.956 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:36 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-06-01T01:55:36.956 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:36 smithi116 ceph-mon[29958]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi161=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:55:36.956 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:36 smithi116 ceph-mon[29958]: Saving service prometheus spec with placement smithi161=a;count:1 2024-06-01T01:55:36.956 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:36 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:36.956 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:36 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:36.956 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:36 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:36.956 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:36 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:55:36.956 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:36 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' 2024-06-01T01:55:36.956 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:36 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-06-01T01:55:37.196 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:36 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:55:36.957+0000 7f650c723000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-01T01:55:37.196 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:37 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:55:37.056+0000 7f650c723000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-01T01:55:37.294 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:37 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:55:37.055+0000 7ff5ce0b0000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-01T01:55:37.645 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:37 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:55:37.367+0000 7f650c723000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-01T01:55:37.645 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:37 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:55:37.449+0000 7f650c723000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-01T01:55:37.660 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:37 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:55:37.359+0000 7ff5ce0b0000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-01T01:55:37.660 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:37 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:55:37.441+0000 7ff5ce0b0000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-01T01:55:37.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:37 smithi161 ceph-mon[26055]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-06-01T01:55:37.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:37 smithi161 ceph-mon[26055]: mgrmap e16: y(active, since 4m), standbys: x 2024-06-01T01:55:38.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:37 smithi116 ceph-mon[25082]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-06-01T01:55:38.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:37 smithi116 ceph-mon[25082]: mgrmap e16: y(active, since 4m), standbys: x 2024-06-01T01:55:38.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:37 smithi116 ceph-mon[29958]: from='mgr.14152 172.21.15.116:0/1691031934' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-06-01T01:55:38.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:37 smithi116 ceph-mon[29958]: mgrmap e16: y(active, since 4m), standbys: x 2024-06-01T01:55:38.885 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:38 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:55:38.628+0000 7f650c723000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-01T01:55:38.885 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:38 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:55:38.801+0000 7f650c723000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-01T01:55:38.885 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:38 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:55:38.887+0000 7f650c723000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-01T01:55:39.044 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:38 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:55:38.551+0000 7ff5ce0b0000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-01T01:55:39.045 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:38 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:55:38.715+0000 7ff5ce0b0000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-01T01:55:39.045 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:38 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:55:38.797+0000 7ff5ce0b0000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-01T01:55:39.529 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:39 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:55:39.110+0000 7ff5ce0b0000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-01T01:55:39.530 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:39 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:55:39.200+0000 7ff5ce0b0000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-01T01:55:39.530 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:39 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:55:39.292+0000 7ff5ce0b0000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-01T01:55:39.530 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:39 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:55:39.532+0000 7ff5ce0b0000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-01T01:55:39.558 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:39 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:55:39.214+0000 7f650c723000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-01T01:55:39.559 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:39 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:55:39.310+0000 7f650c723000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-01T01:55:39.559 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:39 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:55:39.408+0000 7f650c723000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-01T01:55:39.946 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:39 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:55:39.688+0000 7f650c723000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-01T01:55:40.260 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:39 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:55:39.967+0000 7ff5ce0b0000 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-01T01:55:40.419 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:40 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:55:40.158+0000 7f650c723000 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-01T01:55:40.544 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:40 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:55:40.476+0000 7ff5ce0b0000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-01T01:55:40.686 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:40 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:55:40.650+0000 7f650c723000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-01T01:55:40.858 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:40 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:55:40.569+0000 7ff5ce0b0000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-01T01:55:40.945 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:40 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:55:40.745+0000 7f650c723000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-01T01:55:41.294 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:40 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:55:40.861+0000 7ff5ce0b0000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-01T01:55:41.295 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:40 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:55:40.950+0000 7ff5ce0b0000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-01T01:55:41.446 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:41 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:55:41.040+0000 7f650c723000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-01T01:55:41.446 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:41 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:55:41.129+0000 7f650c723000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-01T01:55:41.685 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:41 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:55:41.352+0000 7ff5ce0b0000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-01T01:55:41.685 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:41 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:55:41.482+0000 7ff5ce0b0000 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-01T01:55:41.685 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:41 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:55:41.574+0000 7ff5ce0b0000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-01T01:55:41.863 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:41 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:55:41.532+0000 7f650c723000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-01T01:55:41.863 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:41 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:55:41.663+0000 7f650c723000 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-01T01:55:41.863 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:41 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:55:41.754+0000 7f650c723000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-01T01:55:41.863 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:41 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:55:41.865+0000 7f650c723000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-01T01:55:42.044 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:41 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:55:41.687+0000 7ff5ce0b0000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-01T01:55:42.045 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:41 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:55:41.778+0000 7ff5ce0b0000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-01T01:55:42.195 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:41 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:55:41.954+0000 7f650c723000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-01T01:55:42.674 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:42 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:55:42.321+0000 7ff5ce0b0000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-01T01:55:42.838 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:42 smithi161 ceph-mon[26055]: Active manager daemon y restarted 2024-06-01T01:55:42.838 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:42 smithi161 ceph-mon[26055]: Activating manager daemon y 2024-06-01T01:55:42.838 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:42 smithi161 ceph-mon[26055]: osdmap e59: 8 total, 8 up, 8 in 2024-06-01T01:55:42.839 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:42 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:55:42.496+0000 7f650c723000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-01T01:55:42.938 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:42 smithi116 ceph-mon[25082]: Active manager daemon y restarted 2024-06-01T01:55:42.938 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:42 smithi116 ceph-mon[25082]: Activating manager daemon y 2024-06-01T01:55:42.938 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:42 smithi116 ceph-mon[25082]: osdmap e59: 8 total, 8 up, 8 in 2024-06-01T01:55:42.938 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:42 smithi116 ceph-mon[29958]: Active manager daemon y restarted 2024-06-01T01:55:42.938 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:42 smithi116 ceph-mon[29958]: Activating manager daemon y 2024-06-01T01:55:42.938 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:42 smithi116 ceph-mon[29958]: osdmap e59: 8 total, 8 up, 8 in 2024-06-01T01:55:42.939 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:42 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:55:42.676+0000 7ff5ce0b0000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-01T01:55:42.939 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:42 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: [01/Jun/2024:01:55:42] ENGINE Bus STARTING 2024-06-01T01:55:43.099 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:42 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:55:42.875+0000 7f650c723000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-01T01:55:43.099 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:42 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: [01/Jun/2024:01:55:42] ENGINE Bus STARTING 2024-06-01T01:55:43.099 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:42 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: CherryPy Checker: 2024-06-01T01:55:43.099 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:42 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: The Application mounted at '' has an empty config. 2024-06-01T01:55:43.099 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:42 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: 2024-06-01T01:55:43.099 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:42 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: [01/Jun/2024:01:55:42] ENGINE Serving on http://:::9283 2024-06-01T01:55:43.099 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:55:42 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: [01/Jun/2024:01:55:42] ENGINE Bus STARTED 2024-06-01T01:55:43.191 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:42 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: CherryPy Checker: 2024-06-01T01:55:43.192 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:42 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: The Application mounted at '' has an empty config. 2024-06-01T01:55:43.192 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:42 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T01:55:43.192 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:42 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: [01/Jun/2024:01:55:42] ENGINE Serving on http://:::9283 2024-06-01T01:55:43.192 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:42 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: [01/Jun/2024:01:55:42] ENGINE Bus STARTED 2024-06-01T01:55:43.544 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: [01/Jun/2024:01:55:43] ENGINE Bus STARTING 2024-06-01T01:55:43.545 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: [01/Jun/2024:01:55:43] ENGINE Serving on https://172.21.15.116:7150 2024-06-01T01:55:43.545 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: [01/Jun/2024:01:55:43] ENGINE Bus STARTED 2024-06-01T01:55:43.774 INFO:teuthology.orchestra.run.smithi161.stdout:Scheduled node-exporter update... 2024-06-01T01:55:44.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: mgrmap e17: y(active, starting, since 0.0607956s), standbys: x 2024-06-01T01:55:44.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T01:55:44.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:55:44.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:55:44.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-01T01:55:44.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-01T01:55:44.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T01:55:44.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T01:55:44.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T01:55:44.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T01:55:44.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T01:55:44.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T01:55:44.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T01:55:44.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T01:55:44.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-01T01:55:44.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-01T01:55:44.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-01T01:55:44.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: Manager daemon y is now available 2024-06-01T01:55:44.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:44.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:44.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:44.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:44.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:55:44.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi116", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:55:44.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:55:44.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi116", "name": "osd_memory_target"}]': finished 2024-06-01T01:55:44.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: Updating smithi116:/etc/ceph/ceph.conf 2024-06-01T01:55:44.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: mgrmap e17: y(active, starting, since 0.0607956s), standbys: x 2024-06-01T01:55:44.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T01:55:44.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:55:44.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:55:44.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-01T01:55:44.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-01T01:55:44.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T01:55:44.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T01:55:44.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T01:55:44.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T01:55:44.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T01:55:44.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T01:55:44.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T01:55:44.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T01:55:44.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-01T01:55:44.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-01T01:55:44.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-01T01:55:44.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: Manager daemon y is now available 2024-06-01T01:55:44.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:44.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:44.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:44.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:44.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:55:44.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi116", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:55:44.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:55:44.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi116", "name": "osd_memory_target"}]': finished 2024-06-01T01:55:44.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: Updating smithi116:/etc/ceph/ceph.conf 2024-06-01T01:55:44.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]': finished 2024-06-01T01:55:44.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: Updating smithi161:/etc/ceph/ceph.conf 2024-06-01T01:55:44.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:44.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T01:55:44.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T01:55:44.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: Standby manager daemon x restarted 2024-06-01T01:55:44.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: Standby manager daemon x started 2024-06-01T01:55:44.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.161:0/1689080013' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-01T01:55:44.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.161:0/1689080013' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T01:55:44.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.161:0/1689080013' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-01T01:55:44.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.161:0/1689080013' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T01:55:44.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: Updating smithi116:/etc/ceph/ceph.client.admin.keyring 2024-06-01T01:55:44.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-06-01T01:55:44.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: [01/Jun/2024:01:55:43] ENGINE Bus STARTING 2024-06-01T01:55:44.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: [01/Jun/2024:01:55:43] ENGINE Serving on https://172.21.15.116:7150 2024-06-01T01:55:44.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: [01/Jun/2024:01:55:43] ENGINE Bus STARTED 2024-06-01T01:55:44.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:44.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:44.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:44.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:44.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-06-01T01:55:44.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]': finished 2024-06-01T01:55:44.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: Updating smithi161:/etc/ceph/ceph.conf 2024-06-01T01:55:44.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:44.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T01:55:44.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T01:55:44.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: Standby manager daemon x restarted 2024-06-01T01:55:44.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: Standby manager daemon x started 2024-06-01T01:55:44.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.161:0/1689080013' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-01T01:55:44.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.161:0/1689080013' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T01:55:44.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.161:0/1689080013' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-01T01:55:44.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.161:0/1689080013' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T01:55:44.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: Updating smithi116:/etc/ceph/ceph.client.admin.keyring 2024-06-01T01:55:44.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-06-01T01:55:44.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: [01/Jun/2024:01:55:43] ENGINE Bus STARTING 2024-06-01T01:55:44.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: [01/Jun/2024:01:55:43] ENGINE Serving on https://172.21.15.116:7150 2024-06-01T01:55:44.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: [01/Jun/2024:01:55:43] ENGINE Bus STARTED 2024-06-01T01:55:44.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:44.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:44.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:44.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:44.054 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-06-01T01:55:44.099 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: mgrmap e17: y(active, starting, since 0.0607956s), standbys: x 2024-06-01T01:55:44.099 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T01:55:44.099 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T01:55:44.099 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T01:55:44.099 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-01T01:55:44.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-01T01:55:44.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T01:55:44.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T01:55:44.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T01:55:44.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T01:55:44.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T01:55:44.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T01:55:44.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T01:55:44.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T01:55:44.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-01T01:55:44.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-01T01:55:44.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-01T01:55:44.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: Manager daemon y is now available 2024-06-01T01:55:44.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:44.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:44.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:44.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:44.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:55:44.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi116", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:55:44.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-06-01T01:55:44.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi116", "name": "osd_memory_target"}]': finished 2024-06-01T01:55:44.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: Updating smithi116:/etc/ceph/ceph.conf 2024-06-01T01:55:44.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]': finished 2024-06-01T01:55:44.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: Updating smithi161:/etc/ceph/ceph.conf 2024-06-01T01:55:44.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:44.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T01:55:44.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T01:55:44.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: Standby manager daemon x restarted 2024-06-01T01:55:44.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: Standby manager daemon x started 2024-06-01T01:55:44.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.161:0/1689080013' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-01T01:55:44.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.161:0/1689080013' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T01:55:44.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.161:0/1689080013' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-01T01:55:44.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.161:0/1689080013' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T01:55:44.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: Updating smithi116:/etc/ceph/ceph.client.admin.keyring 2024-06-01T01:55:44.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-06-01T01:55:44.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: [01/Jun/2024:01:55:43] ENGINE Bus STARTING 2024-06-01T01:55:44.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: [01/Jun/2024:01:55:43] ENGINE Serving on https://172.21.15.116:7150 2024-06-01T01:55:44.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: [01/Jun/2024:01:55:43] ENGINE Bus STARTED 2024-06-01T01:55:44.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:44.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:44.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:44.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:44.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-06-01T01:55:44.454 DEBUG:teuthology.orchestra.run.smithi116:node-exporter.a> sudo journalctl -f -n 0 -u ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@node-exporter.a.service 2024-06-01T01:55:44.456 DEBUG:teuthology.orchestra.run.smithi161:node-exporter.b> sudo journalctl -f -n 0 -u ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@node-exporter.b.service 2024-06-01T01:55:44.459 INFO:tasks.cephadm:Adding alertmanager.a on smithi116 2024-06-01T01:55:44.459 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph orch apply alertmanager '1;smithi116=a' 2024-06-01T01:55:45.027 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:44 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-06-01T01:55:45.028 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:44 smithi161 ceph-mon[26055]: mgrmap e18: y(active, since 1.07849s), standbys: x 2024-06-01T01:55:45.028 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:44 smithi161 ceph-mon[26055]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi116=a;smithi161=b", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:55:45.028 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:44 smithi161 ceph-mon[26055]: pgmap v3: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:55:45.028 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:44 smithi161 ceph-mon[26055]: Saving service node-exporter spec with placement smithi116=a;smithi161=b;count:2 2024-06-01T01:55:45.028 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:44 smithi161 ceph-mon[26055]: Deploying daemon prometheus.a on smithi161 2024-06-01T01:55:45.028 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:44 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:45.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:44 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-06-01T01:55:45.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:44 smithi116 ceph-mon[29958]: mgrmap e18: y(active, since 1.07849s), standbys: x 2024-06-01T01:55:45.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:44 smithi116 ceph-mon[29958]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi116=a;smithi161=b", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:55:45.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:44 smithi116 ceph-mon[29958]: pgmap v3: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:55:45.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:44 smithi116 ceph-mon[29958]: Saving service node-exporter spec with placement smithi116=a;smithi161=b;count:2 2024-06-01T01:55:45.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:44 smithi116 ceph-mon[29958]: Deploying daemon prometheus.a on smithi161 2024-06-01T01:55:45.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:44 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:45.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:44 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-06-01T01:55:45.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:44 smithi116 ceph-mon[25082]: mgrmap e18: y(active, since 1.07849s), standbys: x 2024-06-01T01:55:45.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:44 smithi116 ceph-mon[25082]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi116=a;smithi161=b", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:55:45.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:44 smithi116 ceph-mon[25082]: pgmap v3: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:55:45.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:44 smithi116 ceph-mon[25082]: Saving service node-exporter spec with placement smithi116=a;smithi161=b;count:2 2024-06-01T01:55:45.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:44 smithi116 ceph-mon[25082]: Deploying daemon prometheus.a on smithi161 2024-06-01T01:55:45.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:44 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:46.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:46 smithi161 ceph-mon[26055]: pgmap v4: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:55:46.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:46 smithi161 ceph-mon[26055]: mgrmap e19: y(active, since 2s), standbys: x 2024-06-01T01:55:46.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:46 smithi116 ceph-mon[25082]: pgmap v4: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:55:46.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:46 smithi116 ceph-mon[25082]: mgrmap e19: y(active, since 2s), standbys: x 2024-06-01T01:55:46.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:46 smithi116 ceph-mon[29958]: pgmap v4: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:55:46.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:46 smithi116 ceph-mon[29958]: mgrmap e19: y(active, since 2s), standbys: x 2024-06-01T01:55:46.796 INFO:teuthology.orchestra.run.smithi161.stdout:Scheduled alertmanager update... 2024-06-01T01:55:47.810 DEBUG:teuthology.orchestra.run.smithi116:alertmanager.a> sudo journalctl -f -n 0 -u ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@alertmanager.a.service 2024-06-01T01:55:47.813 INFO:tasks.cephadm:Adding grafana.a on smithi161 2024-06-01T01:55:47.814 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph orch apply grafana '1;smithi161=a' 2024-06-01T01:55:48.122 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:47 smithi161 ceph-mon[26055]: pgmap v5: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:55:48.122 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:47 smithi161 ceph-mon[26055]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi116=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:55:48.122 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:47 smithi161 ceph-mon[26055]: Saving service alertmanager spec with placement smithi116=a;count:1 2024-06-01T01:55:48.122 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:47 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:48.192 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:47 smithi116 ceph-mon[29958]: pgmap v5: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:55:48.192 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:47 smithi116 ceph-mon[29958]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi116=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:55:48.192 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:47 smithi116 ceph-mon[29958]: Saving service alertmanager spec with placement smithi116=a;count:1 2024-06-01T01:55:48.193 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:47 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:48.193 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:47 smithi116 ceph-mon[25082]: pgmap v5: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:55:48.193 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:47 smithi116 ceph-mon[25082]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi116=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:55:48.193 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:47 smithi116 ceph-mon[25082]: Saving service alertmanager spec with placement smithi116=a;count:1 2024-06-01T01:55:48.193 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:47 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:50.305 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:50 smithi116 ceph-mon[29958]: pgmap v6: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:55:50.305 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:50 smithi116 ceph-mon[25082]: pgmap v6: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:55:50.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:50 smithi161 ceph-mon[26055]: pgmap v6: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:55:51.001 INFO:teuthology.orchestra.run.smithi161.stdout:Scheduled grafana update... 2024-06-01T01:55:52.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:52 smithi116 ceph-mon[25082]: pgmap v7: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:55:52.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:52 smithi116 ceph-mon[25082]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi161=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:55:52.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:52 smithi116 ceph-mon[25082]: Saving service grafana spec with placement smithi161=a;count:1 2024-06-01T01:55:52.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:52 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:52.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:52 smithi116 ceph-mon[29958]: pgmap v7: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:55:52.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:52 smithi116 ceph-mon[29958]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi161=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:55:52.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:52 smithi116 ceph-mon[29958]: Saving service grafana spec with placement smithi161=a;count:1 2024-06-01T01:55:52.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:52 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:52.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:52 smithi161 ceph-mon[26055]: pgmap v7: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:55:52.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:52 smithi161 ceph-mon[26055]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi161=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:55:52.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:52 smithi161 ceph-mon[26055]: Saving service grafana spec with placement smithi161=a;count:1 2024-06-01T01:55:52.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:52 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:52.785 DEBUG:teuthology.orchestra.run.smithi161:grafana.a> sudo journalctl -f -n 0 -u ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@grafana.a.service 2024-06-01T01:55:52.787 INFO:tasks.cephadm:Setting up client nodes... 2024-06-01T01:55:52.787 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-06-01T01:55:54.163 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:54 smithi116 ceph-mon[25082]: pgmap v8: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:55:54.163 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:54 smithi116 ceph-mon[29958]: pgmap v8: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:55:54.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:54 smithi161 ceph-mon[26055]: pgmap v8: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:55:54.894 INFO:teuthology.orchestra.run.smithi116.stdout:[client.0] 2024-06-01T01:55:54.894 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQCqf1pmVW5ONRAAQJK9/CMmKpVaq7Rzw8wIqw== 2024-06-01T01:55:55.181 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:55 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/3815737377' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-01T01:55:55.182 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:55 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/3815737377' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-01T01:55:55.252 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:55 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/3815737377' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-01T01:55:55.252 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:55 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/3815737377' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-01T01:55:55.252 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:55 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/3815737377' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-01T01:55:55.252 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:55 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/3815737377' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-01T01:55:55.653 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-06-01T01:55:55.653 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-06-01T01:55:55.653 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-06-01T01:55:55.692 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-06-01T01:55:56.281 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:56 smithi161 ceph-mon[26055]: pgmap v9: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:55:56.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:56 smithi116 ceph-mon[25082]: pgmap v9: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:55:56.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:56 smithi116 ceph-mon[29958]: pgmap v9: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:55:57.938 INFO:teuthology.orchestra.run.smithi161.stdout:[client.1] 2024-06-01T01:55:57.939 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCtf1pmi+PaNxAA5xt006widVKFLH3duiqCKg== 2024-06-01T01:55:58.326 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:58 smithi161 ceph-mon[26055]: pgmap v10: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:55:58.326 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:58 smithi161 ceph-mon[26055]: from='client.? 172.21.15.161:0/860951996' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-01T01:55:58.326 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:58 smithi161 ceph-mon[26055]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-01T01:55:58.326 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:58 smithi161 ceph-mon[26055]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-01T01:55:58.327 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:55:58 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[42329]: ts=2024-06-01T01:55:58.269Z caller=main.go:896 level=info msg="Server is ready to receive web requests." 2024-06-01T01:55:58.327 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:55:58 smithi161 bash[42284]: 12518624b73eea56b495bade8f9c103cc0f78f0832426076c81345bcfdf34c0a 2024-06-01T01:55:58.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:58 smithi116 ceph-mon[25082]: pgmap v10: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:55:58.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:58 smithi116 ceph-mon[25082]: from='client.? 172.21.15.161:0/860951996' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-01T01:55:58.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:58 smithi116 ceph-mon[25082]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-01T01:55:58.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:58 smithi116 ceph-mon[25082]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-01T01:55:58.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:58 smithi116 ceph-mon[29958]: pgmap v10: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:55:58.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:58 smithi116 ceph-mon[29958]: from='client.? 172.21.15.161:0/860951996' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-01T01:55:58.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:58 smithi116 ceph-mon[29958]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-01T01:55:58.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:58 smithi116 ceph-mon[29958]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-01T01:55:58.696 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:55:58 smithi161 systemd[1]: Started Ceph prometheus.a for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T01:55:59.495 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-06-01T01:55:59.496 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-06-01T01:55:59.496 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-06-01T01:55:59.523 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:59 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:59.524 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:59 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:59.524 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:59 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:59.524 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:59 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:55:59.524 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:59 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:59.524 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:59 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:55:59.524 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:59 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-01T01:55:59.524 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:59 smithi161 ceph-mon[26055]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-01T01:55:59.524 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:59 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://172.21.15.161:9095"}]: dispatch 2024-06-01T01:55:59.524 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:59 smithi161 ceph-mon[26055]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://172.21.15.161:9095"}]: dispatch 2024-06-01T01:55:59.524 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:59 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:59.524 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:59 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:59.524 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:59 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:59.524 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:59 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:55:59.525 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:59 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:59.525 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:59 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:55:59.525 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:59 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:59.525 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:59 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:59.525 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:59 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:55:59.525 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:59 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:59.525 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:55:59 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:55:59.541 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-06-01T01:55:59.541 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-06-01T01:55:59.541 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph mgr dump --format=json 2024-06-01T01:55:59.759 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/mon.a/config 2024-06-01T01:55:59.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:59.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:59.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:59.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:55:59.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:59.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:55:59.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-01T01:55:59.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[29958]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-01T01:55:59.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://172.21.15.161:9095"}]: dispatch 2024-06-01T01:55:59.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[29958]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://172.21.15.161:9095"}]: dispatch 2024-06-01T01:55:59.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:59.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:59.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:59.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:55:59.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:59.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:55:59.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:59.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:59.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:55:59.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:59.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:55:59.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:59.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:59.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:59.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:55:59.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:59.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:55:59.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-01T01:55:59.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[25082]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-01T01:55:59.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://172.21.15.161:9095"}]: dispatch 2024-06-01T01:55:59.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[25082]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://172.21.15.161:9095"}]: dispatch 2024-06-01T01:55:59.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:59.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:59.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:59.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:55:59.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:59.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:55:59.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:55:59.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:55:59.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:55:59.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:55:59.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:56:00.190 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:55:59 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:55:59] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:56:00.467 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:00 smithi116 ceph-mon[25082]: pgmap v11: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:00.467 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:00 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:00.467 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:00 smithi116 ceph-mon[29958]: pgmap v11: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:00.467 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:00 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:00.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:00 smithi161 ceph-mon[26055]: pgmap v11: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:00.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:00 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:01.647 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-06-01T01:56:02.445 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":19,"active_gid":14427,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":189859058},{"type":"v1","addr":"172.21.15.116:6801","nonce":189859058}]},"active_addr":"172.21.15.116:6801/189859058","active_change":"2024-06-01T01:55:42.679492+0000","active_mgr_features":4540138303579357183,"available":true,"standbys":[{"gid":24323,"name":"x","mgr_features":4540138303579357183,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.116:8443/","prometheus":"http://172.21.15.116:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":59,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.116:0","nonce":3243331104}]},{"addrvec":[{"type":"v2","addr":"172.21.15.116:0","nonce":1699237466}]},{"addrvec":[{"type":"v2","addr":"172.21.15.116:0","nonce":1833534794}]},{"addrvec":[{"type":"v2","addr":"172.21.15.116:0","nonce":2988728965}]}]}} 2024-06-01T01:56:02.451 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-06-01T01:56:02.451 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-06-01T01:56:02.451 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-01T01:56:02.651 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:02 smithi161 ceph-mon[26055]: pgmap v12: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:02.651 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:02 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/4087882924' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-01T01:56:02.665 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/mon.a/config 2024-06-01T01:56:02.787 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:02 smithi116 ceph-mon[25082]: pgmap v12: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:02.787 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:02 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/4087882924' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-01T01:56:02.787 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:02 smithi116 ceph-mon[29958]: pgmap v12: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:02.787 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:02 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/4087882924' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-01T01:56:03.294 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:56:03 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:56:03] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:56:04.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:03 smithi116 ceph-mon[25082]: pgmap v13: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:04.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:03 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:04.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:03 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:04.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:03 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:04.045 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:56:03 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:56:03] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:56:04.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:03 smithi116 ceph-mon[29958]: pgmap v13: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:04.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:03 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:04.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:03 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:04.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:03 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:04.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:03 smithi161 ceph-mon[26055]: pgmap v13: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:04.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:03 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:04.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:03 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:04.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:03 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:04.578 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-06-01T01:56:04.578 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":59,"fsid":"393f964a-1fb9-11ef-bc9b-c7b262605968","created":"2024-06-01T01:50:23.493545+0000","modified":"2024-06-01T01:55:42.679093+0000","last_up_change":"2024-06-01T01:55:23.915950+0000","last_in_change":"2024-06-01T01:55:07.058813+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-01T01:53:25.218001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"77d7478a-9bf5-473d-8296-b0c91060b912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":1460215654},{"type":"v1","addr":"172.21.15.116:6803","nonce":1460215654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":1460215654},{"type":"v1","addr":"172.21.15.116:6805","nonce":1460215654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":1460215654},{"type":"v1","addr":"172.21.15.116:6809","nonce":1460215654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":1460215654},{"type":"v1","addr":"172.21.15.116:6807","nonce":1460215654}]},"public_addr":"172.21.15.116:6803/1460215654","cluster_addr":"172.21.15.116:6805/1460215654","heartbeat_back_addr":"172.21.15.116:6809/1460215654","heartbeat_front_addr":"172.21.15.116:6807/1460215654","state":["exists","up"]},{"osd":1,"uuid":"95343a32-622a-4716-8fcc-71ce91004700","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":166790657},{"type":"v1","addr":"172.21.15.116:6811","nonce":166790657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":166790657},{"type":"v1","addr":"172.21.15.116:6813","nonce":166790657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":166790657},{"type":"v1","addr":"172.21.15.116:6817","nonce":166790657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":166790657},{"type":"v1","addr":"172.21.15.116:6815","nonce":166790657}]},"public_addr":"172.21.15.116:6811/166790657","cluster_addr":"172.21.15.116:6813/166790657","heartbeat_back_addr":"172.21.15.116:6817/166790657","heartbeat_front_addr":"172.21.15.116:6815/166790657","state":["exists","up"]},{"osd":2,"uuid":"d392c3f3-f08e-48d0-aa79-727eb1f9c71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2956549862},{"type":"v1","addr":"172.21.15.116:6819","nonce":2956549862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2956549862},{"type":"v1","addr":"172.21.15.116:6821","nonce":2956549862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2956549862},{"type":"v1","addr":"172.21.15.116:6825","nonce":2956549862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2956549862},{"type":"v1","addr":"172.21.15.116:6823","nonce":2956549862}]},"public_addr":"172.21.15.116:6819/2956549862","cluster_addr":"172.21.15.116:6821/2956549862","heartbeat_back_addr":"172.21.15.116:6825/2956549862","heartbeat_front_addr":"172.21.15.116:6823/2956549862","state":["exists","up"]},{"osd":3,"uuid":"73a18daf-1e3e-476c-8375-96a67d726ef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3228322598},{"type":"v1","addr":"172.21.15.116:6827","nonce":3228322598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3228322598},{"type":"v1","addr":"172.21.15.116:6829","nonce":3228322598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3228322598},{"type":"v1","addr":"172.21.15.116:6833","nonce":3228322598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3228322598},{"type":"v1","addr":"172.21.15.116:6831","nonce":3228322598}]},"public_addr":"172.21.15.116:6827/3228322598","cluster_addr":"172.21.15.116:6829/3228322598","heartbeat_back_addr":"172.21.15.116:6833/3228322598","heartbeat_front_addr":"172.21.15.116:6831/3228322598","state":["exists","up"]},{"osd":4,"uuid":"6b1b3903-2ee8-4483-aeac-e6ffb3eff7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":147846900},{"type":"v1","addr":"172.21.15.161:6801","nonce":147846900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":147846900},{"type":"v1","addr":"172.21.15.161:6803","nonce":147846900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":147846900},{"type":"v1","addr":"172.21.15.161:6807","nonce":147846900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":147846900},{"type":"v1","addr":"172.21.15.161:6805","nonce":147846900}]},"public_addr":"172.21.15.161:6801/147846900","cluster_addr":"172.21.15.161:6803/147846900","heartbeat_back_addr":"172.21.15.161:6807/147846900","heartbeat_front_addr":"172.21.15.161:6805/147846900","state":["exists","up"]},{"osd":5,"uuid":"8f14db41-92df-409b-9fbd-070185f04e0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1993506828},{"type":"v1","addr":"172.21.15.161:6809","nonce":1993506828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1993506828},{"type":"v1","addr":"172.21.15.161:6811","nonce":1993506828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1993506828},{"type":"v1","addr":"172.21.15.161:6815","nonce":1993506828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1993506828},{"type":"v1","addr":"172.21.15.161:6813","nonce":1993506828}]},"public_addr":"172.21.15.161:6809/1993506828","cluster_addr":"172.21.15.161:6811/1993506828","heartbeat_back_addr":"172.21.15.161:6815/1993506828","heartbeat_front_addr":"172.21.15.161:6813/1993506828","state":["exists","up"]},{"osd":6,"uuid":"eefb882a-3b0c-4390-8875-625ef74c717a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3451887914},{"type":"v1","addr":"172.21.15.161:6817","nonce":3451887914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":3451887914},{"type":"v1","addr":"172.21.15.161:6819","nonce":3451887914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":3451887914},{"type":"v1","addr":"172.21.15.161:6823","nonce":3451887914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":3451887914},{"type":"v1","addr":"172.21.15.161:6821","nonce":3451887914}]},"public_addr":"172.21.15.161:6817/3451887914","cluster_addr":"172.21.15.161:6819/3451887914","heartbeat_back_addr":"172.21.15.161:6823/3451887914","heartbeat_front_addr":"172.21.15.161:6821/3451887914","state":["exists","up"]},{"osd":7,"uuid":"704f8550-799e-4d55-9e02-c5ec18173973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2386571128},{"type":"v1","addr":"172.21.15.161:6825","nonce":2386571128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2386571128},{"type":"v1","addr":"172.21.15.161:6827","nonce":2386571128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2386571128},{"type":"v1","addr":"172.21.15.161:6831","nonce":2386571128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2386571128},{"type":"v1","addr":"172.21.15.161:6829","nonce":2386571128}]},"public_addr":"172.21.15.161:6825/2386571128","cluster_addr":"172.21.15.161:6827/2386571128","heartbeat_back_addr":"172.21.15.161:6831/2386571128","heartbeat_front_addr":"172.21.15.161:6829/2386571128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-06-01T01:52:34.917939+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-06-01T01:52:58.344980+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-06-01T01:53:22.397763+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-06-01T01:53:47.760963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-06-01T01:54:11.293430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-06-01T01:54:34.125895+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-06-01T01:54:58.321697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-06-01T01:55:22.013822+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.116:0/3390453904":"2024-06-02T01:55:42.679042+0000","172.21.15.116:0/3769678136":"2024-06-02T01:51:12.859459+0000","172.21.15.116:6801/3557119511":"2024-06-02T01:51:12.859459+0000","172.21.15.116:6801/1936404193":"2024-06-02T01:55:42.679042+0000","172.21.15.116:6800/3557119511":"2024-06-02T01:51:12.859459+0000","172.21.15.116:0/2080484873":"2024-06-02T01:51:12.859459+0000","172.21.15.116:0/3719895177":"2024-06-02T01:55:42.679042+0000","172.21.15.116:0/4034720111":"2024-06-02T01:55:42.679042+0000","172.21.15.116:0/1985344094":"2024-06-02T01:51:12.859459+0000","172.21.15.116:0/455415098":"2024-06-02T01:50:49.481521+0000","172.21.15.116:0/3329906888":"2024-06-02T01:50:49.481521+0000","172.21.15.116:0/3120590326":"2024-06-02T01:50:49.481521+0000","172.21.15.116:6800/1936404193":"2024-06-02T01:55:42.679042+0000","172.21.15.116:0/2550839719":"2024-06-02T01:55:42.679042+0000","172.21.15.116:6800/2138133985":"2024-06-02T01:50:49.481521+0000","172.21.15.116:6801/2138133985":"2024-06-02T01:50:49.481521+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-01T01:56:04.843 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:56:04 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:56:04] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:56:04.843 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:56:04 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:56:04] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:56:05.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:04 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:05.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:04 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/325965521' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-01T01:56:05.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:04 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:05.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:04 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:05.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:04 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:05.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:04 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:56:05.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:04 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:56:05.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:04 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:56:05.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:04 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:05.288 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:04 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:05.289 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:04 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/325965521' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-01T01:56:05.289 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:04 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:05.289 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:04 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:05.289 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:04 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:05.289 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:04 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:56:05.289 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:04 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:56:05.289 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:04 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:56:05.289 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:04 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:05.289 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:04 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:05.289 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:04 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/325965521' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-01T01:56:05.289 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:04 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:05.289 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:04 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:05.289 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:04 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:05.290 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:04 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:56:05.290 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:04 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:56:05.290 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:04 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:56:05.290 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:04 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:05.311 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-06-01T01:56:05.311 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-01T01:56:05.536 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/mon.a/config 2024-06-01T01:56:06.771 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:06 smithi116 ceph-mon[25082]: pgmap v14: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:06.772 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:06 smithi116 ceph-mon[25082]: Deploying daemon node-exporter.a on smithi116 2024-06-01T01:56:06.772 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:06 smithi116 ceph-mon[29958]: pgmap v14: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:06.772 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:06 smithi116 ceph-mon[29958]: Deploying daemon node-exporter.a on smithi116 2024-06-01T01:56:06.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:06 smithi161 ceph-mon[26055]: pgmap v14: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:06.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:06 smithi161 ceph-mon[26055]: Deploying daemon node-exporter.a on smithi116 2024-06-01T01:56:07.462 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-06-01T01:56:07.463 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":59,"fsid":"393f964a-1fb9-11ef-bc9b-c7b262605968","created":"2024-06-01T01:50:23.493545+0000","modified":"2024-06-01T01:55:42.679093+0000","last_up_change":"2024-06-01T01:55:23.915950+0000","last_in_change":"2024-06-01T01:55:07.058813+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-01T01:53:25.218001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"77d7478a-9bf5-473d-8296-b0c91060b912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":1460215654},{"type":"v1","addr":"172.21.15.116:6803","nonce":1460215654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":1460215654},{"type":"v1","addr":"172.21.15.116:6805","nonce":1460215654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":1460215654},{"type":"v1","addr":"172.21.15.116:6809","nonce":1460215654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":1460215654},{"type":"v1","addr":"172.21.15.116:6807","nonce":1460215654}]},"public_addr":"172.21.15.116:6803/1460215654","cluster_addr":"172.21.15.116:6805/1460215654","heartbeat_back_addr":"172.21.15.116:6809/1460215654","heartbeat_front_addr":"172.21.15.116:6807/1460215654","state":["exists","up"]},{"osd":1,"uuid":"95343a32-622a-4716-8fcc-71ce91004700","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":166790657},{"type":"v1","addr":"172.21.15.116:6811","nonce":166790657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":166790657},{"type":"v1","addr":"172.21.15.116:6813","nonce":166790657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":166790657},{"type":"v1","addr":"172.21.15.116:6817","nonce":166790657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":166790657},{"type":"v1","addr":"172.21.15.116:6815","nonce":166790657}]},"public_addr":"172.21.15.116:6811/166790657","cluster_addr":"172.21.15.116:6813/166790657","heartbeat_back_addr":"172.21.15.116:6817/166790657","heartbeat_front_addr":"172.21.15.116:6815/166790657","state":["exists","up"]},{"osd":2,"uuid":"d392c3f3-f08e-48d0-aa79-727eb1f9c71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2956549862},{"type":"v1","addr":"172.21.15.116:6819","nonce":2956549862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2956549862},{"type":"v1","addr":"172.21.15.116:6821","nonce":2956549862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2956549862},{"type":"v1","addr":"172.21.15.116:6825","nonce":2956549862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2956549862},{"type":"v1","addr":"172.21.15.116:6823","nonce":2956549862}]},"public_addr":"172.21.15.116:6819/2956549862","cluster_addr":"172.21.15.116:6821/2956549862","heartbeat_back_addr":"172.21.15.116:6825/2956549862","heartbeat_front_addr":"172.21.15.116:6823/2956549862","state":["exists","up"]},{"osd":3,"uuid":"73a18daf-1e3e-476c-8375-96a67d726ef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3228322598},{"type":"v1","addr":"172.21.15.116:6827","nonce":3228322598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3228322598},{"type":"v1","addr":"172.21.15.116:6829","nonce":3228322598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3228322598},{"type":"v1","addr":"172.21.15.116:6833","nonce":3228322598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3228322598},{"type":"v1","addr":"172.21.15.116:6831","nonce":3228322598}]},"public_addr":"172.21.15.116:6827/3228322598","cluster_addr":"172.21.15.116:6829/3228322598","heartbeat_back_addr":"172.21.15.116:6833/3228322598","heartbeat_front_addr":"172.21.15.116:6831/3228322598","state":["exists","up"]},{"osd":4,"uuid":"6b1b3903-2ee8-4483-aeac-e6ffb3eff7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":147846900},{"type":"v1","addr":"172.21.15.161:6801","nonce":147846900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":147846900},{"type":"v1","addr":"172.21.15.161:6803","nonce":147846900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":147846900},{"type":"v1","addr":"172.21.15.161:6807","nonce":147846900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":147846900},{"type":"v1","addr":"172.21.15.161:6805","nonce":147846900}]},"public_addr":"172.21.15.161:6801/147846900","cluster_addr":"172.21.15.161:6803/147846900","heartbeat_back_addr":"172.21.15.161:6807/147846900","heartbeat_front_addr":"172.21.15.161:6805/147846900","state":["exists","up"]},{"osd":5,"uuid":"8f14db41-92df-409b-9fbd-070185f04e0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1993506828},{"type":"v1","addr":"172.21.15.161:6809","nonce":1993506828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1993506828},{"type":"v1","addr":"172.21.15.161:6811","nonce":1993506828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1993506828},{"type":"v1","addr":"172.21.15.161:6815","nonce":1993506828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1993506828},{"type":"v1","addr":"172.21.15.161:6813","nonce":1993506828}]},"public_addr":"172.21.15.161:6809/1993506828","cluster_addr":"172.21.15.161:6811/1993506828","heartbeat_back_addr":"172.21.15.161:6815/1993506828","heartbeat_front_addr":"172.21.15.161:6813/1993506828","state":["exists","up"]},{"osd":6,"uuid":"eefb882a-3b0c-4390-8875-625ef74c717a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3451887914},{"type":"v1","addr":"172.21.15.161:6817","nonce":3451887914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":3451887914},{"type":"v1","addr":"172.21.15.161:6819","nonce":3451887914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":3451887914},{"type":"v1","addr":"172.21.15.161:6823","nonce":3451887914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":3451887914},{"type":"v1","addr":"172.21.15.161:6821","nonce":3451887914}]},"public_addr":"172.21.15.161:6817/3451887914","cluster_addr":"172.21.15.161:6819/3451887914","heartbeat_back_addr":"172.21.15.161:6823/3451887914","heartbeat_front_addr":"172.21.15.161:6821/3451887914","state":["exists","up"]},{"osd":7,"uuid":"704f8550-799e-4d55-9e02-c5ec18173973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2386571128},{"type":"v1","addr":"172.21.15.161:6825","nonce":2386571128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2386571128},{"type":"v1","addr":"172.21.15.161:6827","nonce":2386571128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2386571128},{"type":"v1","addr":"172.21.15.161:6831","nonce":2386571128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2386571128},{"type":"v1","addr":"172.21.15.161:6829","nonce":2386571128}]},"public_addr":"172.21.15.161:6825/2386571128","cluster_addr":"172.21.15.161:6827/2386571128","heartbeat_back_addr":"172.21.15.161:6831/2386571128","heartbeat_front_addr":"172.21.15.161:6829/2386571128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-06-01T01:52:34.917939+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-06-01T01:52:58.344980+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-06-01T01:53:22.397763+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-06-01T01:53:47.760963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-06-01T01:54:11.293430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-06-01T01:54:34.125895+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-06-01T01:54:58.321697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-06-01T01:55:22.013822+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.116:0/3390453904":"2024-06-02T01:55:42.679042+0000","172.21.15.116:0/3769678136":"2024-06-02T01:51:12.859459+0000","172.21.15.116:6801/3557119511":"2024-06-02T01:51:12.859459+0000","172.21.15.116:6801/1936404193":"2024-06-02T01:55:42.679042+0000","172.21.15.116:6800/3557119511":"2024-06-02T01:51:12.859459+0000","172.21.15.116:0/2080484873":"2024-06-02T01:51:12.859459+0000","172.21.15.116:0/3719895177":"2024-06-02T01:55:42.679042+0000","172.21.15.116:0/4034720111":"2024-06-02T01:55:42.679042+0000","172.21.15.116:0/1985344094":"2024-06-02T01:51:12.859459+0000","172.21.15.116:0/455415098":"2024-06-02T01:50:49.481521+0000","172.21.15.116:0/3329906888":"2024-06-02T01:50:49.481521+0000","172.21.15.116:0/3120590326":"2024-06-02T01:50:49.481521+0000","172.21.15.116:6800/1936404193":"2024-06-02T01:55:42.679042+0000","172.21.15.116:0/2550839719":"2024-06-02T01:55:42.679042+0000","172.21.15.116:6800/2138133985":"2024-06-02T01:50:49.481521+0000","172.21.15.116:6801/2138133985":"2024-06-02T01:50:49.481521+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-01T01:56:07.780 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:07 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/3519502028' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-01T01:56:07.780 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:07 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/3519502028' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-01T01:56:07.780 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:07 smithi116 bash[46068]: Getting image source signatures 2024-06-01T01:56:07.780 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:07 smithi116 bash[46068]: Copying blob sha256:b5db1e299295edf3005515ab7879c1df64a33c185d3a7a23aa4dcaa17d26f7b3 2024-06-01T01:56:07.781 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:07 smithi116 bash[46068]: Copying blob sha256:aa2a8d90b84cb2a9c422e7005cd166a008ccf22ef5d7d4f07128478585ce35ea 2024-06-01T01:56:07.781 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:07 smithi116 bash[46068]: Copying blob sha256:b45d31ee2d7f9f452678a85b0c837c29e12089f31ee8dbac6c8c24dfa4054a30 2024-06-01T01:56:07.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:07 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/3519502028' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-01T01:56:08.463 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-06-01T01:56:08.464 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-06-01T01:56:08.464 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-06-01T01:56:08.464 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-06-01T01:56:08.465 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-06-01T01:56:08.465 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-06-01T01:56:08.466 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-06-01T01:56:08.466 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-06-01T01:56:08.691 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:08 smithi116 ceph-mon[29958]: pgmap v15: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:08.692 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:08 smithi116 ceph-mon[25082]: pgmap v15: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:08.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:08 smithi161 ceph-mon[26055]: pgmap v15: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:09.981 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:09 smithi116 bash[46068]: Copying config sha256:1dbe0e931976487e20e5cfb272087e08a9779c88fd5e9617ed7042dd9751ec26 2024-06-01T01:56:09.981 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:09 smithi116 bash[46068]: Writing manifest to image destination 2024-06-01T01:56:10.212 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/mon.a/config 2024-06-01T01:56:10.213 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/mon.a/config 2024-06-01T01:56:10.214 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/mon.a/config 2024-06-01T01:56:10.217 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/mon.a/config 2024-06-01T01:56:10.220 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/mon.a/config 2024-06-01T01:56:10.220 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/mon.a/config 2024-06-01T01:56:10.222 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/mon.a/config 2024-06-01T01:56:10.597 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:10 smithi116 podman[46068]: 2024-06-01 01:56:10.256997793 +0000 UTC m=+2.949813480 image pull 1dbe0e931976487e20e5cfb272087e08a9779c88fd5e9617ed7042dd9751ec26 quay.io/prometheus/node-exporter:v1.3.1 2024-06-01T01:56:10.658 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/mon.a/config 2024-06-01T01:56:10.936 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:10 smithi116 ceph-mon[25082]: pgmap v16: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:10.937 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:10 smithi116 ceph-mon[29958]: pgmap v16: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:10.937 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:10 smithi116 podman[46068]: 2024-06-01 01:56:10.808880472 +0000 UTC m=+3.501696157 container create 35a72a0577d412362116fcfa118e4b15a15303e0d5d2152a2ede4ea54e3022a4 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-06-01T01:56:10.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:10 smithi161 ceph-mon[26055]: pgmap v16: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:11.967 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 podman[46068]: 2024-06-01 01:56:11.86943551 +0000 UTC m=+4.562251207 container init 35a72a0577d412362116fcfa118e4b15a15303e0d5d2152a2ede4ea54e3022a4 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-06-01T01:56:11.968 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 podman[46068]: 2024-06-01 01:56:11.872675732 +0000 UTC m=+4.565491427 container start 35a72a0577d412362116fcfa118e4b15a15303e0d5d2152a2ede4ea54e3022a4 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-06-01T01:56:11.968 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.876Z caller=node_exporter.go:182 level=info msg="Starting node_exporter" version="(version=1.3.1, branch=HEAD, revision=a2321e7b940ddcff26873612bccdf7cd4c42b6b6)" 2024-06-01T01:56:11.968 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.876Z caller=node_exporter.go:183 level=info msg="Build context" build_context="(go=go1.17.3, user=root@243aafa5525c, date=20211205-11:09:49)" 2024-06-01T01:56:11.968 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+)($|/) 2024-06-01T01:56:11.968 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-06-01T01:56:11.968 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:108 level=info msg="Enabled collectors" 2024-06-01T01:56:11.968 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=arp 2024-06-01T01:56:11.968 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=bcache 2024-06-01T01:56:11.968 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=bonding 2024-06-01T01:56:11.968 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=btrfs 2024-06-01T01:56:11.968 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=conntrack 2024-06-01T01:56:11.968 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=cpu 2024-06-01T01:56:11.969 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=cpufreq 2024-06-01T01:56:11.969 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=diskstats 2024-06-01T01:56:11.969 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=dmi 2024-06-01T01:56:11.969 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=edac 2024-06-01T01:56:11.969 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=entropy 2024-06-01T01:56:11.969 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=fibrechannel 2024-06-01T01:56:11.969 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=filefd 2024-06-01T01:56:11.970 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=filesystem 2024-06-01T01:56:11.970 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=hwmon 2024-06-01T01:56:11.970 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=infiniband 2024-06-01T01:56:11.970 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=ipvs 2024-06-01T01:56:11.970 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=loadavg 2024-06-01T01:56:11.970 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=mdadm 2024-06-01T01:56:11.970 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=meminfo 2024-06-01T01:56:11.970 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=netclass 2024-06-01T01:56:11.970 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=netdev 2024-06-01T01:56:11.970 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=netstat 2024-06-01T01:56:11.970 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=nfs 2024-06-01T01:56:11.970 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=nfsd 2024-06-01T01:56:11.970 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=nvme 2024-06-01T01:56:11.971 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=os 2024-06-01T01:56:11.971 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=powersupplyclass 2024-06-01T01:56:11.971 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=pressure 2024-06-01T01:56:11.971 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=rapl 2024-06-01T01:56:11.971 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=schedstat 2024-06-01T01:56:11.971 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=sockstat 2024-06-01T01:56:11.971 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=softnet 2024-06-01T01:56:11.971 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=stat 2024-06-01T01:56:11.971 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=tapestats 2024-06-01T01:56:11.971 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=textfile 2024-06-01T01:56:11.971 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=thermal_zone 2024-06-01T01:56:11.971 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=time 2024-06-01T01:56:11.971 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=udp_queues 2024-06-01T01:56:11.971 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=uname 2024-06-01T01:56:11.971 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=vmstat 2024-06-01T01:56:11.972 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=xfs 2024-06-01T01:56:11.972 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:115 level=info collector=zfs 2024-06-01T01:56:11.972 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=node_exporter.go:199 level=info msg="Listening on" address=:9100 2024-06-01T01:56:11.972 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[46606]: ts=2024-06-01T01:56:11.877Z caller=tls_config.go:195 level=info msg="TLS is disabled." http2=false 2024-06-01T01:56:12.728 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:12 smithi116 ceph-mon[29958]: pgmap v17: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:12.729 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:12 smithi116 ceph-mon[25082]: pgmap v17: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:12.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:12 smithi161 ceph-mon[26055]: pgmap v17: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:14.393 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:14 smithi116 bash[46068]: 35a72a0577d412362116fcfa118e4b15a15303e0d5d2152a2ede4ea54e3022a4 2024-06-01T01:56:14.393 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 01:56:14 smithi116 systemd[1]: Started Ceph node-exporter.a for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T01:56:14.675 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:14 smithi161 ceph-mon[26055]: pgmap v18: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:14.676 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:14 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:14.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:14 smithi116 ceph-mon[25082]: pgmap v18: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:14.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:14 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:14.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:14 smithi116 ceph-mon[29958]: pgmap v18: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:14.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:14 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:15.236 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:15 smithi161 systemd[1]: Starting Ceph node-exporter.b for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T01:56:15.616 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:15 smithi161 bash[43187]: Trying to pull quay.io/prometheus/node-exporter:v1.3.1... 2024-06-01T01:56:15.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:15 smithi161 ceph-mon[26055]: Deploying daemon node-exporter.b on smithi161 2024-06-01T01:56:15.946 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:15 smithi161 bash[43187]: Getting image source signatures 2024-06-01T01:56:15.946 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:15 smithi161 bash[43187]: Copying blob sha256:b5db1e299295edf3005515ab7879c1df64a33c185d3a7a23aa4dcaa17d26f7b3 2024-06-01T01:56:15.946 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:15 smithi161 bash[43187]: Copying blob sha256:aa2a8d90b84cb2a9c422e7005cd166a008ccf22ef5d7d4f07128478585ce35ea 2024-06-01T01:56:15.946 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:15 smithi161 bash[43187]: Copying blob sha256:b45d31ee2d7f9f452678a85b0c837c29e12089f31ee8dbac6c8c24dfa4054a30 2024-06-01T01:56:15.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:15 smithi116 ceph-mon[25082]: Deploying daemon node-exporter.b on smithi161 2024-06-01T01:56:15.969 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:15 smithi116 ceph-mon[29958]: Deploying daemon node-exporter.b on smithi161 2024-06-01T01:56:16.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:16 smithi161 ceph-mon[26055]: pgmap v19: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:16.982 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:16 smithi116 ceph-mon[29958]: pgmap v19: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:16.983 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:16 smithi116 ceph-mon[25082]: pgmap v19: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:18.348 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 bash[43187]: Copying config sha256:1dbe0e931976487e20e5cfb272087e08a9779c88fd5e9617ed7042dd9751ec26 2024-06-01T01:56:18.619 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 bash[43187]: Writing manifest to image destination 2024-06-01T01:56:18.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-mon[26055]: pgmap v20: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:18.912 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 podman[43187]: 2024-06-01 01:56:18.606581452 +0000 UTC m=+3.462041538 image pull 1dbe0e931976487e20e5cfb272087e08a9779c88fd5e9617ed7042dd9751ec26 quay.io/prometheus/node-exporter:v1.3.1 2024-06-01T01:56:18.912 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 podman[43187]: 2024-06-01 01:56:18.713194063 +0000 UTC m=+3.568654178 container create 4456eaae965b95cef5ae97c9c41d5e27da0aaac4ad51e214d2f8cd4a95ecda12 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-06-01T01:56:18.985 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:18 smithi116 ceph-mon[25082]: pgmap v20: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:18.985 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:18 smithi116 ceph-mon[29958]: pgmap v20: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:19.196 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 podman[43187]: 2024-06-01 01:56:18.915596322 +0000 UTC m=+3.771056408 container init 4456eaae965b95cef5ae97c9c41d5e27da0aaac4ad51e214d2f8cd4a95ecda12 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-06-01T01:56:19.196 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 podman[43187]: 2024-06-01 01:56:18.919065606 +0000 UTC m=+3.774525680 container start 4456eaae965b95cef5ae97c9c41d5e27da0aaac4ad51e214d2f8cd4a95ecda12 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-06-01T01:56:19.196 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.924Z caller=node_exporter.go:182 level=info msg="Starting node_exporter" version="(version=1.3.1, branch=HEAD, revision=a2321e7b940ddcff26873612bccdf7cd4c42b6b6)" 2024-06-01T01:56:19.196 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.924Z caller=node_exporter.go:183 level=info msg="Build context" build_context="(go=go1.17.3, user=root@243aafa5525c, date=20211205-11:09:49)" 2024-06-01T01:56:19.197 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+)($|/) 2024-06-01T01:56:19.197 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-06-01T01:56:19.197 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:108 level=info msg="Enabled collectors" 2024-06-01T01:56:19.197 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=arp 2024-06-01T01:56:19.197 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=bcache 2024-06-01T01:56:19.197 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=bonding 2024-06-01T01:56:19.197 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=btrfs 2024-06-01T01:56:19.197 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=conntrack 2024-06-01T01:56:19.197 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=cpu 2024-06-01T01:56:19.197 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=cpufreq 2024-06-01T01:56:19.197 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=diskstats 2024-06-01T01:56:19.198 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=dmi 2024-06-01T01:56:19.198 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=edac 2024-06-01T01:56:19.198 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=entropy 2024-06-01T01:56:19.198 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=fibrechannel 2024-06-01T01:56:19.198 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=filefd 2024-06-01T01:56:19.198 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=filesystem 2024-06-01T01:56:19.199 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=hwmon 2024-06-01T01:56:19.199 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=infiniband 2024-06-01T01:56:19.199 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=ipvs 2024-06-01T01:56:19.199 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=loadavg 2024-06-01T01:56:19.199 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=mdadm 2024-06-01T01:56:19.199 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=meminfo 2024-06-01T01:56:19.199 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=netclass 2024-06-01T01:56:19.199 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=netdev 2024-06-01T01:56:19.199 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=netstat 2024-06-01T01:56:19.199 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=nfs 2024-06-01T01:56:19.199 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=nfsd 2024-06-01T01:56:19.199 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=nvme 2024-06-01T01:56:19.200 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=os 2024-06-01T01:56:19.200 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=powersupplyclass 2024-06-01T01:56:19.200 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=pressure 2024-06-01T01:56:19.200 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=rapl 2024-06-01T01:56:19.200 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=schedstat 2024-06-01T01:56:19.200 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=sockstat 2024-06-01T01:56:19.200 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=softnet 2024-06-01T01:56:19.200 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=stat 2024-06-01T01:56:19.200 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=tapestats 2024-06-01T01:56:19.200 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=textfile 2024-06-01T01:56:19.200 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=thermal_zone 2024-06-01T01:56:19.200 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=time 2024-06-01T01:56:19.200 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=udp_queues 2024-06-01T01:56:19.200 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=uname 2024-06-01T01:56:19.201 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=vmstat 2024-06-01T01:56:19.201 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=xfs 2024-06-01T01:56:19.201 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.925Z caller=node_exporter.go:115 level=info collector=zfs 2024-06-01T01:56:19.201 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.926Z caller=node_exporter.go:199 level=info msg="Listening on" address=:9100 2024-06-01T01:56:19.201 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[43236]: ts=2024-06-01T01:56:18.926Z caller=tls_config.go:195 level=info msg="TLS is disabled." http2=false 2024-06-01T01:56:19.201 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:19 smithi161 bash[43187]: 4456eaae965b95cef5ae97c9c41d5e27da0aaac4ad51e214d2f8cd4a95ecda12 2024-06-01T01:56:19.201 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 01:56:19 smithi161 systemd[1]: Started Ceph node-exporter.b for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T01:56:20.254 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:20 smithi116 ceph-mon[29958]: pgmap v21: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:20.254 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:20 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:20.254 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:20 smithi116 ceph-mon[29958]: Deploying daemon alertmanager.a on smithi116 2024-06-01T01:56:20.254 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:20 smithi116 ceph-mon[25082]: pgmap v21: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:20.255 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:20 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:20.255 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:20 smithi116 ceph-mon[25082]: Deploying daemon alertmanager.a on smithi116 2024-06-01T01:56:20.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:20 smithi161 ceph-mon[26055]: pgmap v21: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:20.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:20 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:20.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:20 smithi161 ceph-mon[26055]: Deploying daemon alertmanager.a on smithi116 2024-06-01T01:56:22.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:22 smithi161 ceph-mon[26055]: pgmap v22: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:22.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:22 smithi116 ceph-mon[25082]: pgmap v22: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:22.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:22 smithi116 ceph-mon[29958]: pgmap v22: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:22.396 INFO:teuthology.orchestra.run.smithi116.stdout:128849018907 2024-06-01T01:56:22.397 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-06-01T01:56:22.885 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:56:22 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:56:22] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:56:23.294 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:56:23 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:56:23] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:56:23.692 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:23 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:23.692 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:23 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:23.692 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:23 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:24.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:23 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:24.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:23 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:24.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:23 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:24.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:23 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:24.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:23 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:24.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:23 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:24.721 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:24 smithi116 ceph-mon[25082]: pgmap v23: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:24.721 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:24 smithi116 ceph-mon[29958]: pgmap v23: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:24.945 INFO:teuthology.orchestra.run.smithi116.stdout:77309411366 2024-06-01T01:56:24.945 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-06-01T01:56:24.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:24 smithi161 ceph-mon[26055]: pgmap v23: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:24.971 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:56:24 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:56:24] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:56:25.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:25 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:25.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:25 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:25.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:25 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:25.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:25 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:26.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:25 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:26.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:25 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:26.517 INFO:teuthology.orchestra.run.smithi116.stdout:107374182433 2024-06-01T01:56:26.517 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-06-01T01:56:26.746 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:26 smithi116 ceph-mon[29958]: pgmap v24: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:26.746 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:26 smithi116 ceph-mon[25082]: pgmap v24: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:27.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:26 smithi161 ceph-mon[26055]: pgmap v24: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:28.115 INFO:teuthology.orchestra.run.smithi116.stdout:163208757272 2024-06-01T01:56:28.115 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-06-01T01:56:29.006 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:28 smithi116 ceph-mon[29958]: pgmap v25: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:29.007 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:28 smithi116 ceph-mon[25082]: pgmap v25: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:29.031 INFO:teuthology.orchestra.run.smithi116.stdout:55834574891 2024-06-01T01:56:29.032 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-06-01T01:56:29.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:28 smithi161 ceph-mon[26055]: pgmap v25: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:29.868 INFO:teuthology.orchestra.run.smithi116.stdout:231928233999 2024-06-01T01:56:29.869 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-06-01T01:56:30.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:30 smithi116 ceph-mon[25082]: pgmap v26: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:30.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:30 smithi116 ceph-mon[29958]: pgmap v26: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:30.351 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/mon.a/config 2024-06-01T01:56:30.351 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/mon.a/config 2024-06-01T01:56:30.351 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/mon.a/config 2024-06-01T01:56:30.364 INFO:teuthology.orchestra.run.smithi116.stdout:34359738416 2024-06-01T01:56:30.364 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-06-01T01:56:30.413 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/mon.a/config 2024-06-01T01:56:30.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:30 smithi161 ceph-mon[26055]: pgmap v26: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:32.101 INFO:teuthology.orchestra.run.smithi116.stdout:197568495634 2024-06-01T01:56:32.101 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-06-01T01:56:32.327 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:32 smithi116 ceph-mon[25082]: pgmap v27: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:32.327 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:32 smithi116 ceph-mon[29958]: pgmap v27: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:32.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:32 smithi161 ceph-mon[26055]: pgmap v27: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:34.415 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:34 smithi116 ceph-mon[29958]: pgmap v28: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:34.416 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:34 smithi116 ceph-mon[25082]: pgmap v28: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:34.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:34 smithi161 ceph-mon[26055]: pgmap v28: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:36.091 INFO:teuthology.orchestra.run.smithi116.stdout:55834574893 2024-06-01T01:56:36.404 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:36 smithi116 ceph-mon[29958]: pgmap v29: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:36.404 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:36 smithi116 ceph-mon[25082]: pgmap v29: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:36.435 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/mon.a/config 2024-06-01T01:56:36.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:36 smithi161 ceph-mon[26055]: pgmap v29: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:36.754 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/mon.a/config 2024-06-01T01:56:36.755 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/mon.a/config 2024-06-01T01:56:37.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:37 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/65180115' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-01T01:56:37.462 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:37 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/65180115' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-01T01:56:37.463 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:37 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/65180115' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-01T01:56:38.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:38 smithi161 ceph-mon[26055]: pgmap v30: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:38.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:38 smithi116 ceph-mon[25082]: pgmap v30: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:38.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:38 smithi116 ceph-mon[29958]: pgmap v30: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:39.826 INFO:teuthology.orchestra.run.smithi116.stdout:107374182435 2024-06-01T01:56:40.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:40 smithi116 ceph-mon[25082]: pgmap v31: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:40.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:40 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1532156637' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-01T01:56:40.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:40 smithi116 ceph-mon[29958]: pgmap v31: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:40.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:40 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1532156637' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-01T01:56:40.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:40 smithi161 ceph-mon[26055]: pgmap v31: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:40.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:40 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1532156637' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-01T01:56:41.156 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182433 got 107374182435 for osd.3 2024-06-01T01:56:41.157 DEBUG:teuthology.parallel:result is None 2024-06-01T01:56:41.294 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:56:41 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:56:41] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:56:41.550 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/mon.a/config 2024-06-01T01:56:41.577 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574891 got 55834574893 for osd.1 2024-06-01T01:56:41.577 DEBUG:teuthology.parallel:result is None 2024-06-01T01:56:41.980 INFO:teuthology.orchestra.run.smithi116.stdout:163208757275 2024-06-01T01:56:42.366 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:42 smithi161 ceph-mon[26055]: pgmap v32: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:42.366 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:42 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:42.366 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:42 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:42.367 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:42 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/3082150289' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-01T01:56:42.490 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:42 smithi116 ceph-mon[29958]: pgmap v32: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:42.490 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:42 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:42.490 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:42 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:42.490 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:42 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/3082150289' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-01T01:56:42.491 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:42 smithi116 ceph-mon[25082]: pgmap v32: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:42.491 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:42 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:42.491 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:42 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:42.491 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:42 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/3082150289' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-01T01:56:43.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T01:56:43.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T01:56:43.494 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T01:56:43.495 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T01:56:43.495 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T01:56:43.495 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T01:56:43.495 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:56:43 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:56:43] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:56:44.285 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:44 smithi116 ceph-mon[25082]: pgmap v33: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:44.285 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:44 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:44.285 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:44 smithi116 ceph-mon[29958]: pgmap v33: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:44.285 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:44 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:44.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:44 smithi161 ceph-mon[26055]: pgmap v33: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:44.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:44 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:46.290 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:46 smithi116 ceph-mon[25082]: pgmap v34: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:46.291 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:46 smithi116 ceph-mon[29958]: pgmap v34: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:46.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:46 smithi161 ceph-mon[26055]: pgmap v34: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:46.878 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757272 got 163208757275 for osd.5 2024-06-01T01:56:46.878 DEBUG:teuthology.parallel:result is None 2024-06-01T01:56:47.964 INFO:teuthology.orchestra.run.smithi116.stdout:128849018912 2024-06-01T01:56:48.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:48 smithi116 ceph-mon[25082]: pgmap v35: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:48.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:48 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1082943129' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-01T01:56:48.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:48 smithi116 ceph-mon[29958]: pgmap v35: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:48.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:48 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1082943129' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-01T01:56:48.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:48 smithi161 ceph-mon[26055]: pgmap v35: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:48.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:48 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1082943129' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-01T01:56:49.157 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018907 got 128849018912 for osd.4 2024-06-01T01:56:49.157 DEBUG:teuthology.parallel:result is None 2024-06-01T01:56:49.624 INFO:teuthology.orchestra.run.smithi116.stdout:77309411371 2024-06-01T01:56:50.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:50 smithi116 ceph-mon[25082]: pgmap v36: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:50.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:50 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/3276135383' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-01T01:56:50.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:50 smithi116 ceph-mon[29958]: pgmap v36: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:50.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:50 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/3276135383' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-01T01:56:50.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:50 smithi161 ceph-mon[26055]: pgmap v36: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:50.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:50 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/3276135383' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-01T01:56:51.584 INFO:teuthology.orchestra.run.smithi116.stdout:231928234004 2024-06-01T01:56:51.795 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:56:51 smithi116 podman[48953]: 2024-06-01 01:56:51.547460717 +0000 UTC m=+0.122529343 image pull ba2b418f427c0636d654de8757e830c80168e76482bcc46bb2138e569d6c91d4 quay.io/prometheus/alertmanager:v0.23.0 2024-06-01T01:56:52.281 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:56:51 smithi116 podman[48953]: 2024-06-01 01:56:51.996092948 +0000 UTC m=+0.571161625 volume create 1dab2c6ec3283f8d1cd263402a8372f982aa124c08d167a41663a5c0d4d1ad8b 2024-06-01T01:56:52.281 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:56:52 smithi116 podman[48953]: 2024-06-01 01:56:52.088273139 +0000 UTC m=+0.663341776 container create dcc54688709982cdf0b7983b5cfaf0d22401e98501173c2d289de2677773108e (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-01T01:56:52.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:52 smithi116 ceph-mon[25082]: pgmap v37: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:52.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:52 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1386414122' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-01T01:56:52.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:52 smithi116 ceph-mon[29958]: pgmap v37: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:52.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:52 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1386414122' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-01T01:56:52.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:52 smithi161 ceph-mon[26055]: pgmap v37: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:52.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:52 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1386414122' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-01T01:56:53.044 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:56:52 smithi116 podman[48953]: 2024-06-01 01:56:52.659142791 +0000 UTC m=+1.234211419 container init dcc54688709982cdf0b7983b5cfaf0d22401e98501173c2d289de2677773108e (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-01T01:56:53.045 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:56:52 smithi116 podman[48953]: 2024-06-01 01:56:52.663155298 +0000 UTC m=+1.238223916 container start dcc54688709982cdf0b7983b5cfaf0d22401e98501173c2d289de2677773108e (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-01T01:56:53.045 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:56:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[48997]: level=info ts=2024-06-01T01:56:52.674Z caller=main.go:225 msg="Starting Alertmanager" version="(version=0.23.0, branch=HEAD, revision=61046b17771a57cfd4c4a51be370ab930a4d7d54)" 2024-06-01T01:56:53.045 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:56:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[48997]: level=info ts=2024-06-01T01:56:52.674Z caller=main.go:226 build_context="(go=go1.16.7, user=root@e21a959be8d2, date=20210825-10:48:55)" 2024-06-01T01:56:53.045 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:56:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[48997]: level=info ts=2024-06-01T01:56:52.675Z caller=cluster.go:184 component=cluster msg="setting advertise address explicitly" addr=172.21.15.116 port=9094 2024-06-01T01:56:53.045 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:56:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[48997]: level=info ts=2024-06-01T01:56:52.677Z caller=cluster.go:671 component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-06-01T01:56:53.045 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:56:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[48997]: level=info ts=2024-06-01T01:56:52.703Z caller=coordinator.go:113 component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-06-01T01:56:53.045 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:56:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[48997]: level=info ts=2024-06-01T01:56:52.704Z caller=coordinator.go:126 component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-06-01T01:56:53.045 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:56:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[48997]: level=info ts=2024-06-01T01:56:52.706Z caller=main.go:518 msg=Listening address=:9093 2024-06-01T01:56:53.045 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:56:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[48997]: level=info ts=2024-06-01T01:56:52.706Z caller=tls_config.go:191 msg="TLS is disabled." http2=false 2024-06-01T01:56:53.045 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:56:52 smithi116 bash[48953]: dcc54688709982cdf0b7983b5cfaf0d22401e98501173c2d289de2677773108e 2024-06-01T01:56:53.045 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:56:52 smithi116 systemd[1]: Started Ceph alertmanager.a for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T01:56:53.534 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411366 got 77309411371 for osd.2 2024-06-01T01:56:53.534 DEBUG:teuthology.parallel:result is None 2024-06-01T01:56:53.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:53 smithi161 ceph-mon[26055]: pgmap v38: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:53.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:53 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:53.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:53 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:53.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:53 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:53.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:53 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-01T01:56:53.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:53 smithi161 ceph-mon[26055]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-01T01:56:53.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:53 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:53.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:53 smithi161 ceph-mon[26055]: Deploying daemon grafana.a on smithi161 2024-06-01T01:56:54.113 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:53 smithi116 ceph-mon[25082]: pgmap v38: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:54.114 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:53 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:54.114 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:53 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:54.114 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:53 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:54.114 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:53 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-01T01:56:54.114 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:53 smithi116 ceph-mon[25082]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-01T01:56:54.114 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:53 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:54.114 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:53 smithi116 ceph-mon[25082]: Deploying daemon grafana.a on smithi161 2024-06-01T01:56:54.114 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:53 smithi116 ceph-mon[29958]: pgmap v38: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:54.115 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:53 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:54.115 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:53 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:54.115 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:53 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:54.115 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:53 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-01T01:56:54.115 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:53 smithi116 ceph-mon[29958]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-01T01:56:54.115 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:53 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:54.115 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:53 smithi116 ceph-mon[29958]: Deploying daemon grafana.a on smithi161 2024-06-01T01:56:54.284 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928233999 got 231928234004 for osd.7 2024-06-01T01:56:54.284 DEBUG:teuthology.parallel:result is None 2024-06-01T01:56:54.744 INFO:teuthology.orchestra.run.smithi116.stdout:197568495639 2024-06-01T01:56:55.045 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:56:54 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[48997]: level=info ts=2024-06-01T01:56:54.678Z caller=cluster.go:696 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000680787s 2024-06-01T01:56:55.356 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:55 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/118798176' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-01T01:56:55.356 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:55 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/118798176' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-01T01:56:55.380 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495634 got 197568495639 for osd.6 2024-06-01T01:56:55.380 DEBUG:teuthology.parallel:result is None 2024-06-01T01:56:55.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:55 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/118798176' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-01T01:56:56.026 INFO:teuthology.orchestra.run.smithi116.stdout:34359738422 2024-06-01T01:56:56.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:56 smithi116 ceph-mon[29958]: pgmap v39: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:56.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:56 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/772520502' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-01T01:56:56.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:56 smithi116 ceph-mon[25082]: pgmap v39: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:56.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:56 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/772520502' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-01T01:56:56.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:56 smithi161 ceph-mon[26055]: pgmap v39: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:56.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:56 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/772520502' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-01T01:56:57.490 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738416 got 34359738422 for osd.0 2024-06-01T01:56:57.491 DEBUG:teuthology.parallel:result is None 2024-06-01T01:56:57.491 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-01T01:56:57.491 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-01T01:56:57.728 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:56:57 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:56:57] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:56:58.182 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/mon.a/config 2024-06-01T01:56:58.603 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:58 smithi116 ceph-mon[25082]: pgmap v40: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:58.603 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:58 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:58.603 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:56:58 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:58.604 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:58 smithi116 ceph-mon[29958]: pgmap v40: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:58.604 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:58 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:58.604 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:56:58 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:58.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:58 smithi161 ceph-mon[26055]: pgmap v40: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:56:58.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:58 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:58.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:56:58 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:56:59.736 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:56:59 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:56:59] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:57:00.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:00 smithi116 ceph-mon[25082]: pgmap v41: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:00.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:00 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:00.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:00 smithi116 ceph-mon[29958]: pgmap v41: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:00.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:00 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:00.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:00 smithi161 ceph-mon[26055]: pgmap v41: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:00.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:00 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:01.363 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:57:01 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:57:01] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:57:01.654 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-06-01T01:57:01.654 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-06-01T01:57:02.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:02 smithi116 ceph-mon[25082]: pgmap v42: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:02.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:02 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:02.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:02 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:02.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:02 smithi116 ceph-mon[25082]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-01T01:57:02.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:02 smithi116 ceph-mon[29958]: pgmap v42: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:02.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:02 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:02.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:02 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:02.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:02 smithi116 ceph-mon[29958]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-01T01:57:02.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:02 smithi161 ceph-mon[26055]: pgmap v42: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:02.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:02 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:02.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:02 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:02.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:02 smithi161 ceph-mon[26055]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-01T01:57:02.930 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:57:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[48997]: level=info ts=2024-06-01T01:57:02.680Z caller=cluster.go:688 component=cluster msg="gossip settled; proceeding" elapsed=10.002811489s 2024-06-01T01:57:03.380 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-06-01T01:57:00.758984+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":8,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":749961216,"kb_used":48776,"kb_used_data":6088,"kb_used_omap":0,"kb_used_meta":42624,"kb_avail":749912440,"statfs":{"total":767960285184,"available":767910338560,"internally_reserved":0,"allocated":6234112,"data_stored":3488336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":43646976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002148"},"pg_stats":[{"pgid":"1.0","version":"59'128","reported_seq":266,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-06-01T01:55:42.828821+0000","last_change":"2024-06-01T01:55:27.079238+0000","last_active":"2024-06-01T01:55:42.828821+0000","last_peered":"2024-06-01T01:55:42.828821+0000","last_clean":"2024-06-01T01:55:42.828821+0000","last_became_active":"2024-06-01T01:55:27.078757+0000","last_became_peered":"2024-06-01T01:55:27.078757+0000","last_unstale":"2024-06-01T01:55:42.828821+0000","last_undegraded":"2024-06-01T01:55:42.828821+0000","last_fullsized":"2024-06-01T01:55:42.828821+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-01T01:53:25.229537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-01T01:53:25.229537+0000","last_clean_scrub_stamp":"2024-06-01T01:53:25.229537+0000","objects_scrubbed":0,"log_size":128,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-02T05:03:36.127416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234006,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6088,"kb_used_data":1088,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":93739064,"statfs":{"total":95995035648,"available":95988801536,"internally_reserved":0,"allocated":1114112,"data_stored":766769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jun 1 01:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.69199999999999995}]},{"osd":1,"last update":"Sat Jun 1 01:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.69899999999999995}]},{"osd":2,"last update":"Sat Jun 1 01:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.71199999999999997}]},{"osd":3,"last update":"Sat Jun 1 01:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.72799999999999998}]},{"osd":4,"last update":"Sat Jun 1 01:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.443},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.46800000000000003}]},{"osd":5,"last update":"Sat Jun 1 01:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.48099999999999998}]},{"osd":6,"last update":"Sat Jun 1 01:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.5}]}]},{"osd":6,"up_from":46,"seq":197568495641,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6148,"kb_used_data":1084,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":93739004,"statfs":{"total":95995035648,"available":95988740096,"internally_reserved":0,"allocated":1110016,"data_stored":766446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jun 1 01:56:00 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.377}]},{"osd":1,"last update":"Sat Jun 1 01:56:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.49099999999999999}]},{"osd":2,"last update":"Sat Jun 1 01:56:00 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.40799999999999997}]},{"osd":3,"last update":"Sat Jun 1 01:56:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.51600000000000001}]},{"osd":4,"last update":"Sat Jun 1 01:56:00 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.28599999999999998}]},{"osd":5,"last update":"Sat Jun 1 01:56:00 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.38400000000000001}]},{"osd":7,"last update":"Sat Jun 1 01:56:23 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.46400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6336,"kb_used_data":568,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738816,"statfs":{"total":95995035648,"available":95988547584,"internally_reserved":0,"allocated":581632,"data_stored":237841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jun 1 01:56:04 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.35299999999999998,"15min":0.33200000000000002},"min":{"1min":0.25,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.52800000000000002,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.35499999999999998,"15min":0.33600000000000002},"min":{"1min":0.24399999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.19500000000000001}]},{"osd":2,"last update":"Sat Jun 1 01:56:28 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.39200000000000002,"15min":0.38300000000000001},"min":{"1min":0.28799999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.54400000000000004,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.40799999999999997,"15min":0.39700000000000002},"min":{"1min":0.27000000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.58299999999999996,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.26100000000000001}]},{"osd":3,"last update":"Sat Jun 1 01:56:54 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.41699999999999998,"15min":0.42099999999999999},"min":{"1min":0.29699999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.47799999999999998,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.371,"5min":0.437,"15min":0.44800000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.49199999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.32500000000000001}]},{"osd":4,"last update":"Sat Jun 1 01:56:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.61699999999999999,"15min":0.61899999999999999},"min":{"1min":0.45300000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.316},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59899999999999998,"15min":0.60199999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83899999999999997,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.376}]},{"osd":5,"last update":"Sat Jun 1 01:56:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.61099999999999999,"15min":0.621},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.72099999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.63700000000000001,"15min":0.64600000000000002},"min":{"1min":0.44700000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.77000000000000002,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.5}]},{"osd":6,"last update":"Sat Jun 1 01:56:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.37},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.42399999999999999}]},{"osd":7,"last update":"Sat Jun 1 01:56:25 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.39700000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738423,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6916,"kb_used_data":1084,"kb_used_omap":0,"kb_used_meta":5824,"kb_avail":93738236,"statfs":{"total":95995035648,"available":95987953664,"internally_reserved":0,"allocated":1110016,"data_stored":766504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5963776},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jun 1 01:56:01 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.32600000000000001,"15min":0.30599999999999999},"min":{"1min":0.27100000000000002,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.52500000000000002,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.378,"5min":0.33600000000000002,"15min":0.314},"min":{"1min":0.248,"5min":0.215,"15min":0.215},"max":{"1min":0.56999999999999995,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.433}]},{"osd":2,"last update":"Sat Jun 1 01:56:31 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.378,"15min":0.36599999999999999},"min":{"1min":0.25700000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.57099999999999995,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.38100000000000001,"15min":0.36799999999999999},"min":{"1min":0.253,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.54500000000000004,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.316}]},{"osd":3,"last update":"Sat Jun 1 01:56:57 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.42699999999999999,"15min":0.439},"min":{"1min":0.27600000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.57899999999999996,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.375},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.436,"15min":0.442},"min":{"1min":0.27800000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.58699999999999997,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.38500000000000001}]},{"osd":4,"last update":"Sat Jun 1 01:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53400000000000003,"15min":0.53300000000000003},"min":{"1min":0.36599999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.44},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57499999999999996,"15min":0.57399999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.47599999999999998}]},{"osd":5,"last update":"Sat Jun 1 01:56:38 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.56399999999999995,"15min":0.56799999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.54200000000000004,"15min":0.53900000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.55400000000000005}]},{"osd":6,"last update":"Sat Jun 1 01:56:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.63500000000000001}]},{"osd":7,"last update":"Sat Jun 1 01:56:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.51600000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411374,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6208,"kb_used_data":568,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738944,"statfs":{"total":95995035648,"available":95988678656,"internally_reserved":0,"allocated":581632,"data_stored":237841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jun 1 01:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.35899999999999999,"15min":0.34100000000000003},"min":{"1min":0.27200000000000002,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.55500000000000005,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.36299999999999999,"15min":0.36199999999999999},"min":{"1min":0.26300000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.47799999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.32200000000000001}]},{"osd":1,"last update":"Sat Jun 1 01:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.374,"15min":0.36199999999999999},"min":{"1min":0.25600000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.373,"15min":0.36699999999999999},"min":{"1min":0.23699999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.40100000000000002}]},{"osd":3,"last update":"Sat Jun 1 01:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42999999999999999,"15min":0.42499999999999999},"min":{"1min":0.30199999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.42299999999999999,"15min":0.42699999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.35199999999999998}]},{"osd":4,"last update":"Sat Jun 1 01:56:22 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53000000000000003,"15min":0.52600000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.40600000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.42899999999999999}]},{"osd":5,"last update":"Sat Jun 1 01:56:40 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.54500000000000004,"15min":0.53500000000000003},"min":{"1min":0.35699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.44},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.54900000000000004}]},{"osd":6,"last update":"Sat Jun 1 01:56:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.46899999999999997}]},{"osd":7,"last update":"Sat Jun 1 01:56:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.503},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.51000000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182440,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5696,"kb_used_data":568,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":93739456,"statfs":{"total":95995035648,"available":95989202944,"internally_reserved":0,"allocated":581632,"data_stored":237841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jun 1 01:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40200000000000002,"15min":0.40600000000000003},"min":{"1min":0.28299999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.42599999999999999,"15min":0.434},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.28499999999999998}]},{"osd":1,"last update":"Sat Jun 1 01:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.44900000000000001,"15min":0.46600000000000003},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.35799999999999998}]},{"osd":2,"last update":"Sat Jun 1 01:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.441,"15min":0.44700000000000001},"min":{"1min":0.29199999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.442,"15min":0.45100000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.308}]},{"osd":4,"last update":"Sat Jun 1 01:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.56499999999999995,"15min":0.55600000000000005},"min":{"1min":0.38300000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.58899999999999997,"15min":0.57899999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.41199999999999998}]},{"osd":5,"last update":"Sat Jun 1 01:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60799999999999998,"15min":0.60499999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.628,"15min":0.62},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.47299999999999998}]},{"osd":6,"last update":"Sat Jun 1 01:56:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.48999999999999999}]},{"osd":7,"last update":"Sat Jun 1 01:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.432}]}]},{"osd":4,"up_from":30,"seq":128849018915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5760,"kb_used_data":568,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739392,"statfs":{"total":95995035648,"available":95989137408,"internally_reserved":0,"allocated":581632,"data_stored":237841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jun 1 01:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54900000000000004,"15min":0.55200000000000005},"min":{"1min":0.36699999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.53900000000000003}]},{"osd":1,"last update":"Sat Jun 1 01:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.56100000000000005,"15min":0.57099999999999995},"min":{"1min":0.371,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70799999999999996,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.55500000000000005}]},{"osd":2,"last update":"Sat Jun 1 01:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53700000000000003,"15min":0.54100000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.66200000000000003,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55900000000000005,"15min":0.56200000000000006},"min":{"1min":0.41599999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.49299999999999999}]},{"osd":3,"last update":"Sat Jun 1 01:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.502}]},{"osd":5,"last update":"Sat Jun 1 01:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.40300000000000002,"15min":0.39600000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.70299999999999996,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.434,"15min":0.42799999999999999},"min":{"1min":0.28799999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.372}]},{"osd":6,"last update":"Sat Jun 1 01:56:01 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.39200000000000002}]},{"osd":7,"last update":"Sat Jun 1 01:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.42899999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757279,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5624,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":93739528,"statfs":{"total":95995035648,"available":95989276672,"internally_reserved":0,"allocated":573440,"data_stored":237253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jun 1 01:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.50700000000000001,"15min":0.50600000000000001},"min":{"1min":0.35399999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72499999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.51000000000000001,"15min":0.51200000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.66900000000000004,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.41799999999999998}]},{"osd":1,"last update":"Sat Jun 1 01:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.49299999999999999,"15min":0.49099999999999999},"min":{"1min":0.32900000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.66200000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.378},{"interface":"front","average":{"1min":0.497,"5min":0.51100000000000001,"15min":0.51300000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67600000000000005,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.45900000000000002}]},{"osd":2,"last update":"Sat Jun 1 01:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.374,"5min":0.372,"15min":0.372},"max":{"1min":0.68200000000000005,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.498,"5min":0.53600000000000003,"15min":0.54200000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71899999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.35499999999999998}]},{"osd":3,"last update":"Sat Jun 1 01:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.53600000000000003,"15min":0.54300000000000004},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68700000000000006,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53000000000000003,"15min":0.52900000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71499999999999997,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.39900000000000002}]},{"osd":4,"last update":"Sat Jun 1 01:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.39700000000000002,"15min":0.38900000000000001},"min":{"1min":0.313,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.40799999999999997,"15min":0.39800000000000002},"min":{"1min":0.34200000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.39000000000000001}]},{"osd":6,"last update":"Sat Jun 1 01:56:02 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.434},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.47899999999999998}]},{"osd":7,"last update":"Sat Jun 1 01:56:27 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.48899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-01T01:57:03.382 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-01T01:57:03.854 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/mon.a/config 2024-06-01T01:57:04.035 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:57:03 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:57:03] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:57:04.287 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:04 smithi116 ceph-mon[25082]: pgmap v43: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:04.287 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:04 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:04.287 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:04 smithi116 ceph-mon[29958]: pgmap v43: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:04.287 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:04 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:04.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:04 smithi161 ceph-mon[26055]: pgmap v43: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:04.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:04 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:05.744 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-06-01T01:57:05.745 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-06-01T01:57:06.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-mon[26055]: pgmap v44: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:06.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:06 smithi116 ceph-mon[25082]: pgmap v44: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:06.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:06 smithi116 ceph-mon[29958]: pgmap v44: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:06.777 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":44,"stamp":"2024-06-01T01:57:04.759993+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":8,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":749961216,"kb_used":48776,"kb_used_data":6088,"kb_used_omap":0,"kb_used_meta":42624,"kb_avail":749912440,"statfs":{"total":767960285184,"available":767910338560,"internally_reserved":0,"allocated":6234112,"data_stored":3488336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":43646976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002373"},"pg_stats":[{"pgid":"1.0","version":"59'128","reported_seq":266,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-06-01T01:55:42.828821+0000","last_change":"2024-06-01T01:55:27.079238+0000","last_active":"2024-06-01T01:55:42.828821+0000","last_peered":"2024-06-01T01:55:42.828821+0000","last_clean":"2024-06-01T01:55:42.828821+0000","last_became_active":"2024-06-01T01:55:27.078757+0000","last_became_peered":"2024-06-01T01:55:27.078757+0000","last_unstale":"2024-06-01T01:55:42.828821+0000","last_undegraded":"2024-06-01T01:55:42.828821+0000","last_fullsized":"2024-06-01T01:55:42.828821+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-01T01:53:25.229537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-01T01:53:25.229537+0000","last_clean_scrub_stamp":"2024-06-01T01:53:25.229537+0000","objects_scrubbed":0,"log_size":128,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-02T05:03:36.127416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234007,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6088,"kb_used_data":1088,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":93739064,"statfs":{"total":95995035648,"available":95988801536,"internally_reserved":0,"allocated":1114112,"data_stored":766769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jun 1 01:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.495}]},{"osd":1,"last update":"Sat Jun 1 01:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.438},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.48599999999999999}]},{"osd":2,"last update":"Sat Jun 1 01:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.52300000000000002}]},{"osd":3,"last update":"Sat Jun 1 01:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.51400000000000001}]},{"osd":4,"last update":"Sat Jun 1 01:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.38800000000000001}]},{"osd":5,"last update":"Sat Jun 1 01:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.41299999999999998}]},{"osd":6,"last update":"Sat Jun 1 01:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.503}]}]},{"osd":6,"up_from":46,"seq":197568495642,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6148,"kb_used_data":1084,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":93739004,"statfs":{"total":95995035648,"available":95988740096,"internally_reserved":0,"allocated":1110016,"data_stored":766446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jun 1 01:57:03 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.52600000000000002,"15min":0.52100000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51900000000000002,"15min":0.51800000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.48199999999999998}]},{"osd":1,"last update":"Sat Jun 1 01:57:03 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56299999999999994,"15min":0.56000000000000005},"min":{"1min":0.42999999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54800000000000004,"15min":0.54100000000000004},"min":{"1min":0.42799999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.503}]},{"osd":2,"last update":"Sat Jun 1 01:57:03 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.501,"15min":0.495},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.498,"15min":0.496},"min":{"1min":0.39300000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83599999999999997,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.49399999999999999}]},{"osd":3,"last update":"Sat Jun 1 01:57:03 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.36099999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81799999999999995,"5min":0.875,"15min":0.875},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.52500000000000002,"15min":0.51700000000000002},"min":{"1min":0.42999999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.52900000000000003}]},{"osd":4,"last update":"Sat Jun 1 01:57:03 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.436,"15min":0.42699999999999999},"min":{"1min":0.378,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.42499999999999999,"15min":0.41399999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.39500000000000002}]},{"osd":5,"last update":"Sat Jun 1 01:57:03 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.432,"15min":0.42599999999999999},"min":{"1min":0.34000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.374},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.47399999999999998,"15min":0.46300000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.53700000000000003}]},{"osd":7,"last update":"Sat Jun 1 01:56:23 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.42799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6336,"kb_used_data":568,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738816,"statfs":{"total":95995035648,"available":95988547584,"internally_reserved":0,"allocated":581632,"data_stored":237841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jun 1 01:56:04 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.35299999999999998,"15min":0.33200000000000002},"min":{"1min":0.25,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.52800000000000002,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.35499999999999998,"15min":0.33600000000000002},"min":{"1min":0.24399999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.55800000000000005}]},{"osd":2,"last update":"Sat Jun 1 01:56:28 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.39200000000000002,"15min":0.38300000000000001},"min":{"1min":0.28799999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.54400000000000004,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.40799999999999997,"15min":0.39700000000000002},"min":{"1min":0.27000000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.58299999999999996,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.65600000000000003}]},{"osd":3,"last update":"Sat Jun 1 01:56:54 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.41699999999999998,"15min":0.42099999999999999},"min":{"1min":0.29699999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.47799999999999998,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.371,"5min":0.437,"15min":0.44800000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.49199999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.55100000000000005}]},{"osd":4,"last update":"Sat Jun 1 01:56:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.61699999999999999,"15min":0.61899999999999999},"min":{"1min":0.45300000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59899999999999998,"15min":0.60199999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83899999999999997,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.71999999999999997}]},{"osd":5,"last update":"Sat Jun 1 01:56:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.61099999999999999,"15min":0.621},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.72099999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.63700000000000001,"15min":0.64600000000000002},"min":{"1min":0.44700000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.77000000000000002,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.76000000000000001}]},{"osd":6,"last update":"Sat Jun 1 01:56:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.61499999999999999}]},{"osd":7,"last update":"Sat Jun 1 01:56:25 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.68100000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6916,"kb_used_data":1084,"kb_used_omap":0,"kb_used_meta":5824,"kb_avail":93738236,"statfs":{"total":95995035648,"available":95987953664,"internally_reserved":0,"allocated":1110016,"data_stored":766504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5963776},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jun 1 01:57:02 2024","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.32800000000000001,"15min":0.307},"min":{"1min":0.20799999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.45200000000000001,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.35199999999999998,"15min":0.31900000000000001},"min":{"1min":0.218,"5min":0.215,"15min":0.215},"max":{"1min":0.53700000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.311}]},{"osd":2,"last update":"Sat Jun 1 01:56:31 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.378,"15min":0.36599999999999999},"min":{"1min":0.25700000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.57099999999999995,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.38100000000000001,"15min":0.36799999999999999},"min":{"1min":0.253,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.54500000000000004,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.26300000000000001}]},{"osd":3,"last update":"Sat Jun 1 01:56:57 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.42699999999999999,"15min":0.439},"min":{"1min":0.27600000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.57899999999999996,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.436,"15min":0.442},"min":{"1min":0.27800000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.58699999999999997,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.28199999999999997}]},{"osd":4,"last update":"Sat Jun 1 01:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53400000000000003,"15min":0.53300000000000003},"min":{"1min":0.36599999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57499999999999996,"15min":0.57399999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.46700000000000003}]},{"osd":5,"last update":"Sat Jun 1 01:56:38 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.56399999999999995,"15min":0.56799999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.54200000000000004,"15min":0.53900000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.42499999999999999}]},{"osd":6,"last update":"Sat Jun 1 01:57:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.379}]},{"osd":7,"last update":"Sat Jun 1 01:56:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.436},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.45900000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411375,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6208,"kb_used_data":568,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738944,"statfs":{"total":95995035648,"available":95988678656,"internally_reserved":0,"allocated":581632,"data_stored":237841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jun 1 01:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.35899999999999999,"15min":0.34100000000000003},"min":{"1min":0.27200000000000002,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.55500000000000005,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.36299999999999999,"15min":0.36199999999999999},"min":{"1min":0.26300000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.47799999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.45200000000000001}]},{"osd":1,"last update":"Sat Jun 1 01:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.374,"15min":0.36199999999999999},"min":{"1min":0.25600000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.373,"15min":0.36699999999999999},"min":{"1min":0.23699999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.40500000000000003}]},{"osd":3,"last update":"Sat Jun 1 01:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42999999999999999,"15min":0.42499999999999999},"min":{"1min":0.30199999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.42299999999999999,"15min":0.42699999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.436}]},{"osd":4,"last update":"Sat Jun 1 01:56:22 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53000000000000003,"15min":0.52600000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.40600000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.54400000000000004}]},{"osd":5,"last update":"Sat Jun 1 01:56:40 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.54500000000000004,"15min":0.53500000000000003},"min":{"1min":0.35699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.51900000000000002}]},{"osd":6,"last update":"Sat Jun 1 01:57:03 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39800000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.73899999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.40799999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.52500000000000002}]},{"osd":7,"last update":"Sat Jun 1 01:56:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.58299999999999996}]}]},{"osd":3,"up_from":25,"seq":107374182441,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5696,"kb_used_data":568,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":93739456,"statfs":{"total":95995035648,"available":95989202944,"internally_reserved":0,"allocated":581632,"data_stored":237841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jun 1 01:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40200000000000002,"15min":0.40600000000000003},"min":{"1min":0.28299999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.216},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.42599999999999999,"15min":0.434},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.29999999999999999}]},{"osd":1,"last update":"Sat Jun 1 01:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.44900000000000001,"15min":0.46600000000000003},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.27200000000000002}]},{"osd":2,"last update":"Sat Jun 1 01:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.441,"15min":0.44700000000000001},"min":{"1min":0.29199999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.312},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.442,"15min":0.45100000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.26300000000000001}]},{"osd":4,"last update":"Sat Jun 1 01:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.56499999999999995,"15min":0.55600000000000005},"min":{"1min":0.38300000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.58899999999999997,"15min":0.57899999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.41699999999999998}]},{"osd":5,"last update":"Sat Jun 1 01:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60799999999999998,"15min":0.60499999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.628,"15min":0.62},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.45500000000000002}]},{"osd":6,"last update":"Sat Jun 1 01:57:02 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66900000000000004,"15min":0.67600000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1859999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.66100000000000003,"15min":0.67200000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.002,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.52900000000000003}]},{"osd":7,"last update":"Sat Jun 1 01:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.503},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.441}]}]},{"osd":4,"up_from":30,"seq":128849018916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5760,"kb_used_data":568,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739392,"statfs":{"total":95995035648,"available":95989137408,"internally_reserved":0,"allocated":581632,"data_stored":237841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jun 1 01:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54900000000000004,"15min":0.55200000000000005},"min":{"1min":0.36699999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.34100000000000003}]},{"osd":1,"last update":"Sat Jun 1 01:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.56100000000000005,"15min":0.57099999999999995},"min":{"1min":0.371,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70799999999999996,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.374}]},{"osd":2,"last update":"Sat Jun 1 01:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53700000000000003,"15min":0.54100000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.66200000000000003,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55900000000000005,"15min":0.56200000000000006},"min":{"1min":0.41599999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.45000000000000001}]},{"osd":3,"last update":"Sat Jun 1 01:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.496}]},{"osd":5,"last update":"Sat Jun 1 01:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.40300000000000002,"15min":0.39600000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.70299999999999996,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.434},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.434,"15min":0.42799999999999999},"min":{"1min":0.28799999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.42199999999999999}]},{"osd":6,"last update":"Sat Jun 1 01:57:03 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.47099999999999997,"15min":0.46600000000000003},"min":{"1min":0.33400000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.51100000000000001,"15min":0.503},"min":{"1min":0.35799999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.81799999999999995,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.503}]},{"osd":7,"last update":"Sat Jun 1 01:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.624}]}]},{"osd":5,"up_from":38,"seq":163208757280,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5624,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":93739528,"statfs":{"total":95995035648,"available":95989276672,"internally_reserved":0,"allocated":573440,"data_stored":237253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jun 1 01:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.50700000000000001,"15min":0.50600000000000001},"min":{"1min":0.35399999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72499999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.51000000000000001,"15min":0.51200000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.66900000000000004,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.39500000000000002}]},{"osd":1,"last update":"Sat Jun 1 01:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.49299999999999999,"15min":0.49099999999999999},"min":{"1min":0.32900000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.66200000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.497,"5min":0.51100000000000001,"15min":0.51300000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67600000000000005,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.42099999999999999}]},{"osd":2,"last update":"Sat Jun 1 01:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.374,"5min":0.372,"15min":0.372},"max":{"1min":0.68200000000000005,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.53600000000000003,"15min":0.54200000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71899999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.432}]},{"osd":3,"last update":"Sat Jun 1 01:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.53600000000000003,"15min":0.54300000000000004},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68700000000000006,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.443},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53000000000000003,"15min":0.52900000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71499999999999997,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.47399999999999998}]},{"osd":4,"last update":"Sat Jun 1 01:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.39700000000000002,"15min":0.38900000000000001},"min":{"1min":0.313,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.40799999999999997,"15min":0.39800000000000002},"min":{"1min":0.34200000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.34000000000000002}]},{"osd":6,"last update":"Sat Jun 1 01:56:02 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.372}]},{"osd":7,"last update":"Sat Jun 1 01:56:27 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.35999999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-01T01:57:06.778 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-01T01:57:06.779 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-06-01T01:57:06.779 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-06-01T01:57:06.779 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- ceph health --format=json 2024-06-01T01:57:06.957 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_snapshot table v4" 2024-06-01T01:57:06.958 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="drop table dashboard_snapshot_v4 #1" 2024-06-01T01:57:06.958 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_snapshot table v5 #2" 2024-06-01T01:57:07.208 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_dashboard_snapshot_key - v5" 2024-06-01T01:57:07.208 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-06-01T01:57:07.208 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_snapshot_user_id - v5" 2024-06-01T01:57:07.209 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="alter dashboard_snapshot to mediumtext v2" 2024-06-01T01:57:07.209 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard_snapshot table charset" 2024-06-01T01:57:07.209 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="Add column external_delete_url to dashboard_snapshots table" 2024-06-01T01:57:07.209 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="Add encrypted dashboard json column" 2024-06-01T01:57:07.209 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-06-01T01:57:07.209 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="create quota table v1" 2024-06-01T01:57:07.209 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_quota_org_id_user_id_target - v1" 2024-06-01T01:57:07.209 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="Update quota table charset" 2024-06-01T01:57:07.209 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="create plugin_setting table" 2024-06-01T01:57:07.209 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-06-01T01:57:07.209 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="Add column plugin_version to plugin_settings" 2024-06-01T01:57:07.209 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="Update plugin_setting table charset" 2024-06-01T01:57:07.209 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="create session table" 2024-06-01T01:57:07.210 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table playlist table" 2024-06-01T01:57:07.210 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table playlist_item table" 2024-06-01T01:57:07.210 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="create playlist table v2" 2024-06-01T01:57:07.211 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="create playlist item table v2" 2024-06-01T01:57:07.211 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="Update playlist table charset" 2024-06-01T01:57:07.211 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="Update playlist_item table charset" 2024-06-01T01:57:07.211 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="drop preferences table v2" 2024-06-01T01:57:07.211 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="drop preferences table v3" 2024-06-01T01:57:07.211 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="create preferences table v3" 2024-06-01T01:57:07.211 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="Update preferences table charset" 2024-06-01T01:57:07.211 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="Add column team_id in preferences" 2024-06-01T01:57:07.211 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="Update team_id column values in preferences" 2024-06-01T01:57:07.211 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="Add column week_start in preferences" 2024-06-01T01:57:07.211 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="create alert table v1" 2024-06-01T01:57:07.212 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert org_id & id " 2024-06-01T01:57:07.212 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert state" 2024-06-01T01:57:07.212 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert dashboard_id" 2024-06-01T01:57:07.212 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="Create alert_rule_tag table v1" 2024-06-01T01:57:07.212 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-06-01T01:57:07.212 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-06-01T01:57:07.212 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-06-01T01:57:07.212 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="Create alert_rule_tag table v2" 2024-06-01T01:57:07.212 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-06-01T01:57:07.212 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="copy alert_rule_tag v1 to v2" 2024-06-01T01:57:07.212 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="drop table alert_rule_tag_v1" 2024-06-01T01:57:07.212 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_notification table v1" 2024-06-01T01:57:07.213 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="Add column is_default" 2024-06-01T01:57:07.213 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="Add column frequency" 2024-06-01T01:57:07.213 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="Add column send_reminder" 2024-06-01T01:57:07.213 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="Add column disable_resolve_message" 2024-06-01T01:57:07.213 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert_notification org_id & name" 2024-06-01T01:57:07.213 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert table charset" 2024-06-01T01:57:07.213 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert_notification table charset" 2024-06-01T01:57:07.213 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="create notification_journal table v1" 2024-06-01T01:57:07.213 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="add index notification_journal org_id & alert_id & notifier_id" 2024-06-01T01:57:07.213 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="drop alert_notification_journal" 2024-06-01T01:57:07.213 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_notification_state table v1" 2024-06-01T01:57:07.213 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-06-01T01:57:07.213 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="Add for to alert table" 2024-06-01T01:57:07.213 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="Add column uid in alert_notification" 2024-06-01T01:57:07.214 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="Update uid column values in alert_notification" 2024-06-01T01:57:07.214 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index alert_notification_org_id_uid" 2024-06-01T01:57:07.214 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="Remove unique index org_id_name" 2024-06-01T01:57:07.214 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="Add column secure_settings in alert_notification" 2024-06-01T01:57:07.214 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:06+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert.settings to mediumtext" 2024-06-01T01:57:07.214 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add non-unique index alert_notification_state_alert_id" 2024-06-01T01:57:07.214 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add non-unique index alert_rule_tag_alert_id" 2024-06-01T01:57:07.214 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old annotation table v4" 2024-06-01T01:57:07.214 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="create annotation table v5" 2024-06-01T01:57:07.214 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 0 v3" 2024-06-01T01:57:07.214 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 1 v3" 2024-06-01T01:57:07.214 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 2 v3" 2024-06-01T01:57:07.215 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 3 v3" 2024-06-01T01:57:07.215 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 4 v3" 2024-06-01T01:57:07.215 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Update annotation table charset" 2024-06-01T01:57:07.215 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add column region_id to annotation table" 2024-06-01T01:57:07.215 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Drop category_id index" 2024-06-01T01:57:07.215 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add column tags to annotation table" 2024-06-01T01:57:07.215 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Create annotation_tag table v2" 2024-06-01T01:57:07.215 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index annotation_tag.annotation_id_tag_id" 2024-06-01T01:57:07.215 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-06-01T01:57:07.215 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-06-01T01:57:07.215 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Create annotation_tag table v3" 2024-06-01T01:57:07.215 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-06-01T01:57:07.215 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="copy annotation_tag v2 to v3" 2024-06-01T01:57:07.215 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="drop table annotation_tag_v2" 2024-06-01T01:57:07.216 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert annotations and set TEXT to empty" 2024-06-01T01:57:07.216 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add created time to annotation table" 2024-06-01T01:57:07.216 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add updated time to annotation table" 2024-06-01T01:57:07.216 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for created in annotation table" 2024-06-01T01:57:07.216 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for updated in annotation table" 2024-06-01T01:57:07.216 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Convert existing annotations from seconds to milliseconds" 2024-06-01T01:57:07.216 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add epoch_end column" 2024-06-01T01:57:07.216 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for epoch_end" 2024-06-01T01:57:07.216 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Make epoch_end the same as epoch" 2024-06-01T01:57:07.216 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Move region to single row" 2024-06-01T01:57:07.216 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_epoch from annotation table" 2024-06-01T01:57:07.216 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-06-01T01:57:07.216 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-06-01T01:57:07.216 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for org_id_epoch_end_epoch on annotation table" 2024-06-01T01:57:07.216 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_epoch_epoch_end from annotation table" 2024-06-01T01:57:07.217 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for alert_id on annotation table" 2024-06-01T01:57:07.217 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="create test_data table" 2024-06-01T01:57:07.217 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_version table v1" 2024-06-01T01:57:07.217 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_version.dashboard_id" 2024-06-01T01:57:07.217 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-06-01T01:57:07.217 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Set dashboard version to 1 where 0" 2024-06-01T01:57:07.217 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="save existing dashboard data in dashboard_version table v1" 2024-06-01T01:57:07.217 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="alter dashboard_version.data to mediumtext v1" 2024-06-01T01:57:07.217 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="create team table" 2024-06-01T01:57:07.217 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index team.org_id" 2024-06-01T01:57:07.217 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index team_org_id_name" 2024-06-01T01:57:07.217 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="create team member table" 2024-06-01T01:57:07.217 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_member.org_id" 2024-06-01T01:57:07.218 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index team_member_org_id_team_id_user_id" 2024-06-01T01:57:07.218 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_member.team_id" 2024-06-01T01:57:07.218 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add column email to team table" 2024-06-01T01:57:07.218 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add column external to team_member table" 2024-06-01T01:57:07.218 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add column permission to team_member table" 2024-06-01T01:57:07.218 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard acl table" 2024-06-01T01:57:07.218 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_dashboard_id" 2024-06-01T01:57:07.218 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_acl_dashboard_id_user_id" 2024-06-01T01:57:07.218 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_acl_dashboard_id_team_id" 2024-06-01T01:57:07.218 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_user_id" 2024-06-01T01:57:07.218 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_team_id" 2024-06-01T01:57:07.218 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_org_id_role" 2024-06-01T01:57:07.218 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_permission" 2024-06-01T01:57:07.219 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="save default acl rules in dashboard_acl table" 2024-06-01T01:57:07.219 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="delete acl rules for deleted dashboards and folders" 2024-06-01T01:57:07.219 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="create tag table" 2024-06-01T01:57:07.219 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index tag.key_value" 2024-06-01T01:57:07.219 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="create login attempt table" 2024-06-01T01:57:07.219 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index login_attempt.username" 2024-06-01T01:57:07.219 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_login_attempt_username - v1" 2024-06-01T01:57:07.219 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-06-01T01:57:07.219 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="create login_attempt v2" 2024-06-01T01:57:07.219 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_login_attempt_username - v2" 2024-06-01T01:57:07.219 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="copy login_attempt v1 to v2" 2024-06-01T01:57:07.219 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="drop login_attempt_tmp_qwerty" 2024-06-01T01:57:07.219 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="create user auth table" 2024-06-01T01:57:07.219 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-06-01T01:57:07.220 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="alter user_auth.auth_id to length 190" 2024-06-01T01:57:07.220 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth access token to user_auth" 2024-06-01T01:57:07.220 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth refresh token to user_auth" 2024-06-01T01:57:07.221 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth token type to user_auth" 2024-06-01T01:57:07.221 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth expiry to user_auth" 2024-06-01T01:57:07.221 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add index to user_id column in user_auth" 2024-06-01T01:57:07.221 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="create server_lock table" 2024-06-01T01:57:07.221 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index server_lock.operation_uid" 2024-06-01T01:57:07.221 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="create user auth token table" 2024-06-01T01:57:07.221 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user_auth_token.auth_token" 2024-06-01T01:57:07.221 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user_auth_token.prev_auth_token" 2024-06-01T01:57:07.221 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index user_auth_token.user_id" 2024-06-01T01:57:07.221 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add revoked_at to the user auth token" 2024-06-01T01:57:07.221 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="create cache_data table" 2024-06-01T01:57:07.221 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index cache_data.cache_key" 2024-06-01T01:57:07.222 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="create short_url table v1" 2024-06-01T01:57:07.222 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index short_url.org_id-uid" 2024-06-01T01:57:07.222 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="delete alert_definition table" 2024-06-01T01:57:07.222 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="recreate alert_definition table" 2024-06-01T01:57:07.222 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition on org_id and title columns" 2024-06-01T01:57:07.222 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition on org_id and uid columns" 2024-06-01T01:57:07.222 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_definition table data column to mediumtext in mysql" 2024-06-01T01:57:07.222 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="drop index in alert_definition on org_id and title columns" 2024-06-01T01:57:07.222 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="drop index in alert_definition on org_id and uid columns" 2024-06-01T01:57:07.222 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index in alert_definition on org_id and title columns" 2024-06-01T01:57:07.222 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index in alert_definition on org_id and uid columns" 2024-06-01T01:57:07.222 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add column paused in alert_definition" 2024-06-01T01:57:07.223 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="drop alert_definition table" 2024-06-01T01:57:07.223 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="delete alert_definition_version table" 2024-06-01T01:57:07.223 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="recreate alert_definition_version table" 2024-06-01T01:57:07.223 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-06-01T01:57:07.223 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-06-01T01:57:07.223 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_definition_version table data column to mediumtext in mysql" 2024-06-01T01:57:07.223 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="drop alert_definition_version table" 2024-06-01T01:57:07.223 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_instance table" 2024-06-01T01:57:07.223 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-06-01T01:57:07.223 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_instance table on def_org_id, current_state columns" 2024-06-01T01:57:07.223 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add column current_state_end to alert_instance" 2024-06-01T01:57:07.223 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-06-01T01:57:07.223 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="remove index def_org_id, current_state on alert_instance" 2024-06-01T01:57:07.224 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="rename def_org_id to rule_org_id in alert_instance" 2024-06-01T01:57:07.224 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="rename def_uid to rule_uid in alert_instance" 2024-06-01T01:57:07.224 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-06-01T01:57:07.224 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index rule_org_id, current_state on alert_instance" 2024-06-01T01:57:07.224 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_rule table" 2024-06-01T01:57:07.224 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id and title columns" 2024-06-01T01:57:07.224 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id and uid columns" 2024-06-01T01:57:07.224 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-06-01T01:57:07.224 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_rule table data column to mediumtext in mysql" 2024-06-01T01:57:07.224 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add column for to alert_rule" 2024-06-01T01:57:07.224 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add column annotations to alert_rule" 2024-06-01T01:57:07.224 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add column labels to alert_rule" 2024-06-01T01:57:07.224 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="remove unique index from alert_rule on org_id, title columns" 2024-06-01T01:57:07.225 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-06-01T01:57:07.225 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add dashboard_uid column to alert_rule" 2024-06-01T01:57:07.225 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add panel_id column to alert_rule" 2024-06-01T01:57:07.225 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-06-01T01:57:07.225 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_rule_version table" 2024-06-01T01:57:07.225 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-06-01T01:57:07.225 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-06-01T01:57:07.225 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_rule_version table data column to mediumtext in mysql" 2024-06-01T01:57:07.225 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add column for to alert_rule_version" 2024-06-01T01:57:07.225 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add column annotations to alert_rule_version" 2024-06-01T01:57:07.225 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add column labels to alert_rule_version" 2024-06-01T01:57:07.225 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id=create_alert_configuration_table 2024-06-01T01:57:07.225 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add column default in alert_configuration" 2024-06-01T01:57:07.226 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-06-01T01:57:07.226 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add column org_id in alert_configuration" 2024-06-01T01:57:07.226 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_configuration table on org_id column" 2024-06-01T01:57:07.226 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id=create_ngalert_configuration_table 2024-06-01T01:57:07.226 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index in ngalert_configuration on org_id column" 2024-06-01T01:57:07.226 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="clear migration entry \"remove unified alerting data\"" 2024-06-01T01:57:07.226 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="move dashboard alerts to unified alerting" 2024-06-01T01:57:07.226 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="create library_element table v1" 2024-06-01T01:57:07.226 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index library_element org_id-folder_id-name-kind" 2024-06-01T01:57:07.226 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="create library_element_connection table v1" 2024-06-01T01:57:07.226 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index library_element_connection element_id-kind-connection_id" 2024-06-01T01:57:07.226 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index library_element org_id_uid" 2024-06-01T01:57:07.227 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="clone move dashboard alerts to unified alerting" 2024-06-01T01:57:07.227 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="create data_keys table" 2024-06-01T01:57:07.227 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="create kv_store table v1" 2024-06-01T01:57:07.227 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index kv_store.org_id-namespace-key" 2024-06-01T01:57:07.227 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="update dashboard_uid and panel_id from existing annotations" 2024-06-01T01:57:07.227 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="create permission table" 2024-06-01T01:57:07.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:07 smithi116 ceph-mon[25082]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-01T01:57:07.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:07 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:07.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:07 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:57:07.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:07 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:57:07.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:07 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:57:07.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:07 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:07.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:07 smithi116 ceph-mon[29958]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-01T01:57:07.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:07 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:07.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:07 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:57:07.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:07 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:57:07.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:07 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:57:07.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:07 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:07.461 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-mon[26055]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-01T01:57:07.461 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:07.461 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T01:57:07.461 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:57:07.461 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:57:07.461 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:07.462 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index permission.role_id" 2024-06-01T01:57:07.462 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index role_id_action_scope" 2024-06-01T01:57:07.462 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="create role table" 2024-06-01T01:57:07.462 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add column display_name" 2024-06-01T01:57:07.462 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add column group_name" 2024-06-01T01:57:07.462 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index role.org_id" 2024-06-01T01:57:07.462 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index role_org_id_name" 2024-06-01T01:57:07.462 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index role_org_id_uid" 2024-06-01T01:57:07.463 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="create team role table" 2024-06-01T01:57:07.463 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_role.org_id" 2024-06-01T01:57:07.463 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index team_role_org_id_team_id_role_id" 2024-06-01T01:57:07.463 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_role.team_id" 2024-06-01T01:57:07.463 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="create user role table" 2024-06-01T01:57:07.463 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index user_role.org_id" 2024-06-01T01:57:07.463 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user_role_org_id_user_id_role_id" 2024-06-01T01:57:07.463 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index user_role.user_id" 2024-06-01T01:57:07.463 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="create builtin role table" 2024-06-01T01:57:07.463 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index builtin_role.role_id" 2024-06-01T01:57:07.463 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index builtin_role.name" 2024-06-01T01:57:07.463 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add column org_id to builtin_role table" 2024-06-01T01:57:07.463 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index builtin_role.org_id" 2024-06-01T01:57:07.463 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index builtin_role_org_id_role_id_role" 2024-06-01T01:57:07.464 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="Remove unique index role_org_id_uid" 2024-06-01T01:57:07.464 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index role.uid" 2024-06-01T01:57:07.464 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="create seed assignment table" 2024-06-01T01:57:07.464 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index builtin_role_role_name" 2024-06-01T01:57:07.464 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="migrations completed" logger=migrator performed=381 skipped=0 duration=532.862489ms 2024-06-01T01:57:07.464 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Created default organization" logger=sqlstore 2024-06-01T01:57:07.464 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Initialising plugins" logger=plugin.manager 2024-06-01T01:57:07.464 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=input 2024-06-01T01:57:07.464 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=vonage-status-panel 2024-06-01T01:57:07.464 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=grafana-piechart-panel 2024-06-01T01:57:07.464 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="Live Push Gateway initialization" logger=live.push_http 2024-06-01T01:57:07.464 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=warn msg="[Deprecated] the datasource provisioning config is outdated. please upgrade" logger=provisioning.datasources filename=/etc/grafana/provisioning/datasources/ceph-dashboard.yml 2024-06-01T01:57:07.464 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="inserting datasource from configuration " logger=provisioning.datasources name=Dashboard1 uid=P43CA22E17D0F9596 2024-06-01T01:57:07.464 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="warming cache for startup" logger=ngalert 2024-06-01T01:57:07.464 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="HTTP Server Listen" logger=http.server address=[::]:3000 protocol=https subUrl= socket= 2024-06-01T01:57:07.464 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 01:57:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T01:57:07+0000 lvl=info msg="starting MultiOrg Alertmanager" logger=ngalert.multiorg.alertmanager 2024-06-01T01:57:08.284 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/mon.a/config 2024-06-01T01:57:08.520 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:08 smithi116 ceph-mon[25082]: pgmap v45: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:08.520 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:08 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:57:08.520 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:08 smithi116 ceph-mon[25082]: Reconfiguring alertmanager.a (dependencies changed)... 2024-06-01T01:57:08.520 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:08 smithi116 ceph-mon[25082]: Reconfiguring daemon alertmanager.a on smithi116 2024-06-01T01:57:08.520 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:08 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:08.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:08 smithi116 ceph-mon[29958]: pgmap v45: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:08.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:08 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:57:08.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:08 smithi116 ceph-mon[29958]: Reconfiguring alertmanager.a (dependencies changed)... 2024-06-01T01:57:08.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:08 smithi116 ceph-mon[29958]: Reconfiguring daemon alertmanager.a on smithi116 2024-06-01T01:57:08.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:08 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:08.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:08 smithi161 ceph-mon[26055]: pgmap v45: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:08.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:08 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:57:08.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:08 smithi161 ceph-mon[26055]: Reconfiguring alertmanager.a (dependencies changed)... 2024-06-01T01:57:08.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:08 smithi161 ceph-mon[26055]: Reconfiguring daemon alertmanager.a on smithi116 2024-06-01T01:57:08.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:08 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:09.724 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:57:09 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:57:09] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:57:10.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:10 smithi116 ceph-mon[25082]: pgmap v46: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:10.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:10 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:10.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:10 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:10.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:10 smithi116 ceph-mon[29958]: pgmap v46: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:10.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:10 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:10.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:10 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:10.796 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:10 smithi161 ceph-mon[26055]: pgmap v46: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:10.796 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:10 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:10.796 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:10 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:12.663 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:12 smithi161 ceph-mon[26055]: pgmap v47: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:12.729 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:12 smithi116 ceph-mon[29958]: pgmap v47: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:12.729 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:12 smithi116 ceph-mon[25082]: pgmap v47: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:12.999 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:57:12 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:57:12] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:57:13.209 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-06-01T01:57:13.209 INFO:teuthology.orchestra.run.smithi116.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-06-01T01:57:14.044 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:57:13 smithi116 systemd[1]: Stopping Ceph alertmanager.a for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T01:57:14.045 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:57:13 smithi116 bash[50584]: Error: no container with name or ID "ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager.a" found: no such container 2024-06-01T01:57:14.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:13 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:14.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:13 smithi116 ceph-mon[25082]: pgmap v48: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:14.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:13 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1973875126' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-01T01:57:14.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:13 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:14.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:13 smithi116 ceph-mon[29958]: pgmap v48: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:14.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:13 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1973875126' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-01T01:57:14.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:13 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:14.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:13 smithi161 ceph-mon[26055]: pgmap v48: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:14.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:13 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1973875126' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-01T01:57:14.490 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-06-01T01:57:14.490 INFO:tasks.cephadm:Setup complete, yielding 2024-06-01T01:57:14.491 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-01T01:57:14.502 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi116.front.sepia.ceph.com 2024-06-01T01:57:14.502 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest false --force' 2024-06-01T01:57:14.794 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:57:14 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[48997]: level=info ts=2024-06-01T01:57:14.598Z caller=main.go:557 msg="Received SIGTERM, exiting gracefully..." 2024-06-01T01:57:15.149 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:57:15 smithi116 podman[50596]: 2024-06-01 01:57:15.014731504 +0000 UTC m=+1.266388976 container cleanup dcc54688709982cdf0b7983b5cfaf0d22401e98501173c2d289de2677773108e (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-01T01:57:15.149 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:57:15 smithi116 bash[50596]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a 2024-06-01T01:57:15.149 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:57:15 smithi116 bash[50719]: Error: no container with name or ID "ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager.a" found: no such container 2024-06-01T01:57:15.795 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:57:15 smithi116 podman[50641]: 2024-06-01 01:57:15.493249008 +0000 UTC m=+0.874385268 container remove dcc54688709982cdf0b7983b5cfaf0d22401e98501173c2d289de2677773108e (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-01T01:57:15.795 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:57:15 smithi116 podman[50641]: 2024-06-01 01:57:15.594097723 +0000 UTC m=+0.975233938 volume remove 1dab2c6ec3283f8d1cd263402a8372f982aa124c08d167a41663a5c0d4d1ad8b 2024-06-01T01:57:15.795 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:57:15 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-06-01T01:57:15.795 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:57:15 smithi116 systemd[1]: Stopped Ceph alertmanager.a for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T01:57:15.795 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:57:15 smithi116 systemd[1]: Starting Ceph alertmanager.a for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T01:57:16.269 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:16 smithi116 ceph-mon[25082]: pgmap v49: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:16.270 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:16 smithi116 ceph-mon[29958]: pgmap v49: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:16.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:16 smithi161 ceph-mon[26055]: pgmap v49: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:18.045 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:57:17 smithi116 podman[50946]: 2024-06-01 01:57:17.795114146 +0000 UTC m=+0.202514810 image pull ba2b418f427c0636d654de8757e830c80168e76482bcc46bb2138e569d6c91d4 quay.io/prometheus/alertmanager:v0.23.0 2024-06-01T01:57:18.045 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:57:18 smithi116 podman[50946]: 2024-06-01 01:57:18.000326629 +0000 UTC m=+0.407727282 volume create be2bad1a8e29ad772204f5a766e30a6ba23e6d554dc18e535265dc2fd20be991 2024-06-01T01:57:18.339 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:18 smithi116 ceph-mon[25082]: pgmap v50: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:18.340 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:18 smithi116 ceph-mon[29958]: pgmap v50: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:18.340 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:57:18 smithi116 podman[50946]: 2024-06-01 01:57:18.134473571 +0000 UTC m=+0.541874217 container create 7cd3733dd5843e182b46675bed8d086b6272114cc448b27add914fdd6dddafaf (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-01T01:57:18.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:18 smithi161 ceph-mon[26055]: pgmap v50: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:19.044 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:57:18 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:57:18] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:57:19.045 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:57:18 smithi116 podman[50946]: 2024-06-01 01:57:18.663553238 +0000 UTC m=+1.070953884 container init 7cd3733dd5843e182b46675bed8d086b6272114cc448b27add914fdd6dddafaf (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-01T01:57:19.045 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:57:18 smithi116 podman[50946]: 2024-06-01 01:57:18.667747514 +0000 UTC m=+1.075148158 container start 7cd3733dd5843e182b46675bed8d086b6272114cc448b27add914fdd6dddafaf (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-01T01:57:19.045 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:57:18 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=info ts=2024-06-01T01:57:18.678Z caller=main.go:225 msg="Starting Alertmanager" version="(version=0.23.0, branch=HEAD, revision=61046b17771a57cfd4c4a51be370ab930a4d7d54)" 2024-06-01T01:57:19.045 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:57:18 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=info ts=2024-06-01T01:57:18.678Z caller=main.go:226 build_context="(go=go1.16.7, user=root@e21a959be8d2, date=20210825-10:48:55)" 2024-06-01T01:57:19.045 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:57:18 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=info ts=2024-06-01T01:57:18.679Z caller=cluster.go:184 component=cluster msg="setting advertise address explicitly" addr=172.21.15.116 port=9094 2024-06-01T01:57:19.045 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:57:18 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=info ts=2024-06-01T01:57:18.680Z caller=cluster.go:671 component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-06-01T01:57:19.045 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:57:18 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=info ts=2024-06-01T01:57:18.705Z caller=coordinator.go:113 component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-06-01T01:57:19.045 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:57:18 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=info ts=2024-06-01T01:57:18.705Z caller=coordinator.go:126 component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-06-01T01:57:19.045 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:57:18 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=info ts=2024-06-01T01:57:18.707Z caller=main.go:518 msg=Listening address=:9093 2024-06-01T01:57:19.045 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:57:18 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=info ts=2024-06-01T01:57:18.707Z caller=tls_config.go:191 msg="TLS is disabled." http2=false 2024-06-01T01:57:19.045 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:57:18 smithi116 bash[50946]: 7cd3733dd5843e182b46675bed8d086b6272114cc448b27add914fdd6dddafaf 2024-06-01T01:57:19.046 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:57:18 smithi116 systemd[1]: Started Ceph alertmanager.a for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T01:57:19.245 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-01T01:57:19.253 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi116.front.sepia.ceph.com 2024-06-01T01:57:19.254 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'radosgw-admin realm create --rgw-realm=r --default' 2024-06-01T01:57:19.366 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:19 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/3724438067' entity='client.admin' 2024-06-01T01:57:19.366 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:19 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:19.366 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:19 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:19.366 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:19 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:19.366 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:19 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/3724438067' entity='client.admin' 2024-06-01T01:57:19.366 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:19 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:19.366 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:19 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:19.366 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:19 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:19.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:19 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/3724438067' entity='client.admin' 2024-06-01T01:57:19.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:19 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:19.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:19 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:19.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:19 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:20.245 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:20 smithi116 ceph-mon[29958]: pgmap v51: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:20.245 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:20 smithi116 ceph-mon[29958]: Reconfiguring prometheus.a (dependencies changed)... 2024-06-01T01:57:20.245 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:20 smithi116 ceph-mon[29958]: Reconfiguring daemon prometheus.a on smithi161 2024-06-01T01:57:20.246 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:20 smithi116 ceph-mon[25082]: pgmap v51: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:20.246 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:20 smithi116 ceph-mon[25082]: Reconfiguring prometheus.a (dependencies changed)... 2024-06-01T01:57:20.246 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:20 smithi116 ceph-mon[25082]: Reconfiguring daemon prometheus.a on smithi161 2024-06-01T01:57:20.372 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:20 smithi161 ceph-mon[26055]: pgmap v51: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:20.372 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:20 smithi161 ceph-mon[26055]: Reconfiguring prometheus.a (dependencies changed)... 2024-06-01T01:57:20.372 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:20 smithi161 ceph-mon[26055]: Reconfiguring daemon prometheus.a on smithi161 2024-06-01T01:57:21.045 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:57:20 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=info ts=2024-06-01T01:57:20.680Z caller=cluster.go:696 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000247609s 2024-06-01T01:57:22.219 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:22 smithi161 ceph-mon[26055]: pgmap v52: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:22.219 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:22 smithi161 systemd[1]: Stopping Ceph prometheus.a for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T01:57:22.219 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:22 smithi161 bash[45332]: Error: no container with name or ID "ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus.a" found: no such container 2024-06-01T01:57:22.451 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:22 smithi116 ceph-mon[25082]: pgmap v52: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:22.451 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:22 smithi116 ceph-mon[29958]: pgmap v52: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:22.503 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:22 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[42329]: ts=2024-06-01T01:57:22.221Z caller=main.go:775 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-06-01T01:57:22.503 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:22 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[42329]: ts=2024-06-01T01:57:22.222Z caller=main.go:798 level=info msg="Stopping scrape discovery manager..." 2024-06-01T01:57:22.503 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:22 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[42329]: ts=2024-06-01T01:57:22.222Z caller=main.go:812 level=info msg="Stopping notify discovery manager..." 2024-06-01T01:57:22.503 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:22 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[42329]: ts=2024-06-01T01:57:22.222Z caller=main.go:834 level=info msg="Stopping scrape manager..." 2024-06-01T01:57:22.503 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:22 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[42329]: ts=2024-06-01T01:57:22.222Z caller=manager.go:945 level=info component="rule manager" msg="Stopping rule manager..." 2024-06-01T01:57:22.503 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:22 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[42329]: ts=2024-06-01T01:57:22.222Z caller=main.go:808 level=info msg="Notify discovery manager stopped" 2024-06-01T01:57:22.503 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:22 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[42329]: ts=2024-06-01T01:57:22.222Z caller=main.go:794 level=info msg="Scrape discovery manager stopped" 2024-06-01T01:57:22.503 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:22 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[42329]: ts=2024-06-01T01:57:22.222Z caller=main.go:828 level=info msg="Scrape manager stopped" 2024-06-01T01:57:22.503 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:22 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[42329]: ts=2024-06-01T01:57:22.222Z caller=manager.go:955 level=info component="rule manager" msg="Rule manager stopped" 2024-06-01T01:57:22.503 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:22 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[42329]: ts=2024-06-01T01:57:22.222Z caller=notifier.go:600 level=info component=notifier msg="Stopping notification manager..." 2024-06-01T01:57:22.503 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:22 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[42329]: ts=2024-06-01T01:57:22.222Z caller=main.go:1054 level=info msg="Notifier manager stopped" 2024-06-01T01:57:22.503 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:22 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[42329]: ts=2024-06-01T01:57:22.222Z caller=main.go:1066 level=info msg="See you next time!" 2024-06-01T01:57:22.503 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:22 smithi161 podman[45342]: 2024-06-01 01:57:22.23203161 +0000 UTC m=+0.100323581 container died 12518624b73eea56b495bade8f9c103cc0f78f0832426076c81345bcfdf34c0a (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-01T01:57:22.810 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:22 smithi161 podman[45342]: 2024-06-01 01:57:22.587821647 +0000 UTC m=+0.456113600 container cleanup 12518624b73eea56b495bade8f9c103cc0f78f0832426076c81345bcfdf34c0a (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-01T01:57:22.813 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:22 smithi161 bash[45342]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a 2024-06-01T01:57:23.061 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:22 smithi161 podman[45356]: 2024-06-01 01:57:22.81393562 +0000 UTC m=+0.583616978 container remove 12518624b73eea56b495bade8f9c103cc0f78f0832426076c81345bcfdf34c0a (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-01T01:57:23.061 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:22 smithi161 bash[45368]: Error: no container with name or ID "ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus.a" found: no such container 2024-06-01T01:57:23.062 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:22 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-06-01T01:57:23.062 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:22 smithi161 systemd[1]: Stopped Ceph prometheus.a for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T01:57:23.062 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:22 smithi161 systemd[1]: Starting Ceph prometheus.a for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T01:57:23.147 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T01:57:23.147 INFO:teuthology.orchestra.run.smithi116.stdout: "id": "1ebaea68-0bcb-4c99-839c-d3107430bc58", 2024-06-01T01:57:23.147 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "r", 2024-06-01T01:57:23.147 INFO:teuthology.orchestra.run.smithi116.stdout: "current_period": "1c7bc5b8-908b-4920-9391-420ae5b79983", 2024-06-01T01:57:23.147 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 1 2024-06-01T01:57:23.147 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T01:57:23.252 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:23 smithi116 ceph-mon[29958]: osdmap e60: 8 total, 8 up, 8 in 2024-06-01T01:57:23.252 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:23 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/2604574' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-06-01T01:57:23.252 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:23 smithi116 ceph-mon[29958]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-06-01T01:57:23.252 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:23 smithi116 ceph-mon[25082]: osdmap e60: 8 total, 8 up, 8 in 2024-06-01T01:57:23.253 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:23 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/2604574' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-06-01T01:57:23.253 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:23 smithi116 ceph-mon[25082]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-06-01T01:57:23.320 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:23 smithi161 ceph-mon[26055]: osdmap e60: 8 total, 8 up, 8 in 2024-06-01T01:57:23.320 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:23 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/2604574' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-06-01T01:57:23.321 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:23 smithi161 ceph-mon[26055]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-06-01T01:57:23.321 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:23 smithi161 podman[45433]: 2024-06-01 01:57:23.064920409 +0000 UTC m=+0.093558416 container create bea6bf7e708305690304e13f81d48feca6103d9a6752f57a0a7b4a21ab34c93a (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-01T01:57:23.321 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:23 smithi161 podman[45433]: 2024-06-01 01:57:22.989095011 +0000 UTC m=+0.017733029 image pull 514e6a882f6e74806a5856468489eeff8d7106095557578da96935e4d0ba4d9d quay.io/prometheus/prometheus:v2.33.4 2024-06-01T01:57:23.321 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:23 smithi161 podman[45433]: 2024-06-01 01:57:23.267186074 +0000 UTC m=+0.295824082 container init bea6bf7e708305690304e13f81d48feca6103d9a6752f57a0a7b4a21ab34c93a (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-01T01:57:23.321 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:23 smithi161 podman[45433]: 2024-06-01 01:57:23.270991675 +0000 UTC m=+0.299629686 container start bea6bf7e708305690304e13f81d48feca6103d9a6752f57a0a7b4a21ab34c93a (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-01T01:57:23.322 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T01:57:23.294Z caller=main.go:475 level=info msg="No time or size retention was set so using the default time retention" duration=15d 2024-06-01T01:57:23.322 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T01:57:23.294Z caller=main.go:512 level=info msg="Starting Prometheus" version="(version=2.33.4, branch=HEAD, revision=83032011a5d3e6102624fe58241a374a7201fee8)" 2024-06-01T01:57:23.322 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T01:57:23.294Z caller=main.go:517 level=info build_context="(go=go1.17.7, user=root@d13bf69e7be8, date=20220222-16:51:28)" 2024-06-01T01:57:23.322 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T01:57:23.294Z caller=main.go:518 level=info host_details="(Linux 5.14.0-452.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024 x86_64 smithi161 (none))" 2024-06-01T01:57:23.322 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T01:57:23.294Z caller=main.go:519 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-06-01T01:57:23.322 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T01:57:23.294Z caller=main.go:520 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-06-01T01:57:23.322 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T01:57:23.296Z caller=web.go:570 level=info component=web msg="Start listening for connections" address=:9095 2024-06-01T01:57:23.322 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T01:57:23.296Z caller=main.go:923 level=info msg="Starting TSDB ..." 2024-06-01T01:57:23.322 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T01:57:23.297Z caller=tls_config.go:195 level=info component=web msg="TLS is disabled." http2=false 2024-06-01T01:57:23.322 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T01:57:23.300Z caller=head.go:493 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-06-01T01:57:23.322 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T01:57:23.301Z caller=head.go:527 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.581µs 2024-06-01T01:57:23.322 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T01:57:23.301Z caller=head.go:533 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-06-01T01:57:23.322 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T01:57:23.302Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-06-01T01:57:23.323 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T01:57:23.302Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-06-01T01:57:23.323 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T01:57:23.302Z caller=head.go:610 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=40.569µs wal_replay_duration=1.464077ms total_replay_duration=1.532571ms 2024-06-01T01:57:23.323 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T01:57:23.303Z caller=main.go:944 level=info fs_type=XFS_SUPER_MAGIC 2024-06-01T01:57:23.323 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T01:57:23.303Z caller=main.go:947 level=info msg="TSDB started" 2024-06-01T01:57:23.323 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T01:57:23.303Z caller=main.go:1128 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-06-01T01:57:23.323 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T01:57:23.324Z caller=main.go:1165 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=21.080242ms db_storage=658ns remote_storage=1.793µs web_handler=389ns query_engine=850ns scrape=615.353µs scrape_sd=32.428µs notify=25.77µs notify_sd=10.81µs rules=20.044121ms 2024-06-01T01:57:23.323 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T01:57:23.324Z caller=main.go:896 level=info msg="Server is ready to receive web requests." 2024-06-01T01:57:23.583 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:23 smithi161 bash[45433]: bea6bf7e708305690304e13f81d48feca6103d9a6752f57a0a7b4a21ab34c93a 2024-06-01T01:57:23.583 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 01:57:23 smithi161 systemd[1]: Started Ceph prometheus.a for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T01:57:24.023 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'radosgw-admin zonegroup create --rgw-zonegroup=default --master --default' 2024-06-01T01:57:24.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[25082]: pgmap v54: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:24.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[25082]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-06-01T01:57:24.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[25082]: osdmap e61: 8 total, 8 up, 8 in 2024-06-01T01:57:24.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:24.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-01T01:57:24.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[25082]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-01T01:57:24.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://172.21.15.116:9093"}]: dispatch 2024-06-01T01:57:24.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[25082]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://172.21.15.116:9093"}]: dispatch 2024-06-01T01:57:24.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:24.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-01T01:57:24.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[25082]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-01T01:57:24.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-01T01:57:24.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[25082]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-01T01:57:24.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://172.21.15.161:3000"}]: dispatch 2024-06-01T01:57:24.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[25082]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://172.21.15.161:3000"}]: dispatch 2024-06-01T01:57:24.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:24.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:57:24.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:57:24.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:24.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:57:24.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:57:24.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:24.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[29958]: pgmap v54: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:24.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[29958]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-06-01T01:57:24.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[29958]: osdmap e61: 8 total, 8 up, 8 in 2024-06-01T01:57:24.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:24.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-01T01:57:24.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[29958]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-01T01:57:24.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://172.21.15.116:9093"}]: dispatch 2024-06-01T01:57:24.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[29958]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://172.21.15.116:9093"}]: dispatch 2024-06-01T01:57:24.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:24.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-01T01:57:24.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[29958]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-01T01:57:24.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-01T01:57:24.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[29958]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-01T01:57:24.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://172.21.15.161:3000"}]: dispatch 2024-06-01T01:57:24.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[29958]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://172.21.15.161:3000"}]: dispatch 2024-06-01T01:57:24.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:24.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:57:24.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:57:24.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:24.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:57:24.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:57:24.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:24.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:24 smithi161 ceph-mon[26055]: pgmap v54: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:57:24.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:24 smithi161 ceph-mon[26055]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-06-01T01:57:24.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:24 smithi161 ceph-mon[26055]: osdmap e61: 8 total, 8 up, 8 in 2024-06-01T01:57:24.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:24 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:24.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:24 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-01T01:57:24.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:24 smithi161 ceph-mon[26055]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-01T01:57:24.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:24 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://172.21.15.116:9093"}]: dispatch 2024-06-01T01:57:24.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:24 smithi161 ceph-mon[26055]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://172.21.15.116:9093"}]: dispatch 2024-06-01T01:57:24.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:24 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:24.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:24 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-01T01:57:24.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:24 smithi161 ceph-mon[26055]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-01T01:57:24.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:24 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-01T01:57:24.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:24 smithi161 ceph-mon[26055]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-01T01:57:24.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:24 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://172.21.15.161:3000"}]: dispatch 2024-06-01T01:57:24.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:24 smithi161 ceph-mon[26055]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://172.21.15.161:3000"}]: dispatch 2024-06-01T01:57:24.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:24 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:24.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:24 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:57:24.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:24 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:57:24.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:24 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:24.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:24 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:57:24.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:24 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:57:24.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:24 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:25.232 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:25 smithi116 ceph-mon[29958]: osdmap e62: 8 total, 8 up, 8 in 2024-06-01T01:57:25.232 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:25 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:25.232 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:25 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:25.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:25 smithi116 ceph-mon[25082]: osdmap e62: 8 total, 8 up, 8 in 2024-06-01T01:57:25.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:25 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:25.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:25 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:25.233 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:57:24 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:57:24] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:57:25.233 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:57:25 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:57:25] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:57:25.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:25 smithi161 ceph-mon[26055]: osdmap e62: 8 total, 8 up, 8 in 2024-06-01T01:57:25.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:25 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:25.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:25 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:26.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:26 smithi161 ceph-mon[26055]: pgmap v57: 33 pgs: 33 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s wr, 1 op/s 2024-06-01T01:57:26.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:26 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:26.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:26 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:26.516 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:26 smithi116 ceph-mon[25082]: pgmap v57: 33 pgs: 33 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s wr, 1 op/s 2024-06-01T01:57:26.516 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:26 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:26.516 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:26 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:26.516 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:26 smithi116 ceph-mon[29958]: pgmap v57: 33 pgs: 33 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s wr, 1 op/s 2024-06-01T01:57:26.517 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:26 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:26.517 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:26 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:27.386 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T01:57:27.386 INFO:teuthology.orchestra.run.smithi116.stdout: "id": "027b600c-7676-481b-b1c6-6b1a5a28256b", 2024-06-01T01:57:27.386 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "default", 2024-06-01T01:57:27.386 INFO:teuthology.orchestra.run.smithi116.stdout: "api_name": "default", 2024-06-01T01:57:27.386 INFO:teuthology.orchestra.run.smithi116.stdout: "is_master": "true", 2024-06-01T01:57:27.386 INFO:teuthology.orchestra.run.smithi116.stdout: "endpoints": [], 2024-06-01T01:57:27.386 INFO:teuthology.orchestra.run.smithi116.stdout: "hostnames": [], 2024-06-01T01:57:27.386 INFO:teuthology.orchestra.run.smithi116.stdout: "hostnames_s3website": [], 2024-06-01T01:57:27.386 INFO:teuthology.orchestra.run.smithi116.stdout: "master_zone": "", 2024-06-01T01:57:27.387 INFO:teuthology.orchestra.run.smithi116.stdout: "zones": [], 2024-06-01T01:57:27.387 INFO:teuthology.orchestra.run.smithi116.stdout: "placement_targets": [], 2024-06-01T01:57:27.387 INFO:teuthology.orchestra.run.smithi116.stdout: "default_placement": "", 2024-06-01T01:57:27.387 INFO:teuthology.orchestra.run.smithi116.stdout: "realm_id": "1ebaea68-0bcb-4c99-839c-d3107430bc58", 2024-06-01T01:57:27.387 INFO:teuthology.orchestra.run.smithi116.stdout: "sync_policy": { 2024-06-01T01:57:27.387 INFO:teuthology.orchestra.run.smithi116.stdout: "groups": [] 2024-06-01T01:57:27.387 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T01:57:27.387 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T01:57:28.045 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'radosgw-admin zone create --rgw-zonegroup=default --rgw-zone=z --master --default' 2024-06-01T01:57:28.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:28 smithi116 ceph-mon[25082]: pgmap v58: 33 pgs: 33 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s wr, 1 op/s 2024-06-01T01:57:28.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:28 smithi116 ceph-mon[29958]: pgmap v58: 33 pgs: 33 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s wr, 1 op/s 2024-06-01T01:57:28.427 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:28 smithi161 ceph-mon[26055]: pgmap v58: 33 pgs: 33 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s wr, 1 op/s 2024-06-01T01:57:28.761 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:57:28 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=info ts=2024-06-01T01:57:28.682Z caller=cluster.go:688 component=cluster msg="gossip settled; proceeding" elapsed=10.002385336s 2024-06-01T01:57:28.762 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:57:28 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ::ffff:172.21.15.161 - - [01/Jun/2024:01:57:28] "GET /metrics HTTP/1.1" 200 192901 "" "Prometheus/2.33.4" 2024-06-01T01:57:30.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:30 smithi116 ceph-mon[25082]: pgmap v59: 33 pgs: 33 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s wr, 1 op/s 2024-06-01T01:57:30.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:30 smithi116 ceph-mon[29958]: pgmap v59: 33 pgs: 33 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s wr, 1 op/s 2024-06-01T01:57:30.377 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:30 smithi161 ceph-mon[26055]: pgmap v59: 33 pgs: 33 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s wr, 1 op/s 2024-06-01T01:57:31.378 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T01:57:31.378 INFO:teuthology.orchestra.run.smithi116.stdout: "id": "fb01931c-2a09-4e4a-814f-6f0cc6362429", 2024-06-01T01:57:31.378 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "z", 2024-06-01T01:57:31.378 INFO:teuthology.orchestra.run.smithi116.stdout: "domain_root": "z.rgw.meta:root", 2024-06-01T01:57:31.378 INFO:teuthology.orchestra.run.smithi116.stdout: "control_pool": "z.rgw.control", 2024-06-01T01:57:31.378 INFO:teuthology.orchestra.run.smithi116.stdout: "gc_pool": "z.rgw.log:gc", 2024-06-01T01:57:31.378 INFO:teuthology.orchestra.run.smithi116.stdout: "lc_pool": "z.rgw.log:lc", 2024-06-01T01:57:31.378 INFO:teuthology.orchestra.run.smithi116.stdout: "log_pool": "z.rgw.log", 2024-06-01T01:57:31.378 INFO:teuthology.orchestra.run.smithi116.stdout: "intent_log_pool": "z.rgw.log:intent", 2024-06-01T01:57:31.379 INFO:teuthology.orchestra.run.smithi116.stdout: "usage_log_pool": "z.rgw.log:usage", 2024-06-01T01:57:31.379 INFO:teuthology.orchestra.run.smithi116.stdout: "roles_pool": "z.rgw.meta:roles", 2024-06-01T01:57:31.379 INFO:teuthology.orchestra.run.smithi116.stdout: "reshard_pool": "z.rgw.log:reshard", 2024-06-01T01:57:31.379 INFO:teuthology.orchestra.run.smithi116.stdout: "user_keys_pool": "z.rgw.meta:users.keys", 2024-06-01T01:57:31.379 INFO:teuthology.orchestra.run.smithi116.stdout: "user_email_pool": "z.rgw.meta:users.email", 2024-06-01T01:57:31.379 INFO:teuthology.orchestra.run.smithi116.stdout: "user_swift_pool": "z.rgw.meta:users.swift", 2024-06-01T01:57:31.379 INFO:teuthology.orchestra.run.smithi116.stdout: "user_uid_pool": "z.rgw.meta:users.uid", 2024-06-01T01:57:31.379 INFO:teuthology.orchestra.run.smithi116.stdout: "otp_pool": "z.rgw.otp", 2024-06-01T01:57:31.379 INFO:teuthology.orchestra.run.smithi116.stdout: "system_key": { 2024-06-01T01:57:31.379 INFO:teuthology.orchestra.run.smithi116.stdout: "access_key": "", 2024-06-01T01:57:31.379 INFO:teuthology.orchestra.run.smithi116.stdout: "secret_key": "" 2024-06-01T01:57:31.379 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T01:57:31.379 INFO:teuthology.orchestra.run.smithi116.stdout: "placement_pools": [ 2024-06-01T01:57:31.380 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-06-01T01:57:31.380 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "default-placement", 2024-06-01T01:57:31.380 INFO:teuthology.orchestra.run.smithi116.stdout: "val": { 2024-06-01T01:57:31.380 INFO:teuthology.orchestra.run.smithi116.stdout: "index_pool": "z.rgw.buckets.index", 2024-06-01T01:57:31.380 INFO:teuthology.orchestra.run.smithi116.stdout: "storage_classes": { 2024-06-01T01:57:31.380 INFO:teuthology.orchestra.run.smithi116.stdout: "STANDARD": { 2024-06-01T01:57:31.380 INFO:teuthology.orchestra.run.smithi116.stdout: "data_pool": "z.rgw.buckets.data" 2024-06-01T01:57:31.380 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T01:57:31.380 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T01:57:31.380 INFO:teuthology.orchestra.run.smithi116.stdout: "data_extra_pool": "z.rgw.buckets.non-ec", 2024-06-01T01:57:31.380 INFO:teuthology.orchestra.run.smithi116.stdout: "index_type": 0 2024-06-01T01:57:31.380 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T01:57:31.380 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T01:57:31.380 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-06-01T01:57:31.381 INFO:teuthology.orchestra.run.smithi116.stdout: "realm_id": "1ebaea68-0bcb-4c99-839c-d3107430bc58", 2024-06-01T01:57:31.381 INFO:teuthology.orchestra.run.smithi116.stdout: "notif_pool": "z.rgw.log:notif" 2024-06-01T01:57:31.381 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T01:57:32.219 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'radosgw-admin period update --rgw-realm=r --commit' 2024-06-01T01:57:32.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:32 smithi161 ceph-mon[26055]: pgmap v60: 33 pgs: 33 active+clean; 578 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 1.4 KiB/s wr, 2 op/s 2024-06-01T01:57:32.479 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:32 smithi116 ceph-mon[29958]: pgmap v60: 33 pgs: 33 active+clean; 578 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 1.4 KiB/s wr, 2 op/s 2024-06-01T01:57:32.479 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:32 smithi116 ceph-mon[25082]: pgmap v60: 33 pgs: 33 active+clean; 578 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 1.4 KiB/s wr, 2 op/s 2024-06-01T01:57:34.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:34 smithi116 ceph-mon[25082]: pgmap v61: 33 pgs: 33 active+clean; 578 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 424 B/s rd, 1.1 KiB/s wr, 1 op/s 2024-06-01T01:57:34.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:34 smithi116 ceph-mon[29958]: pgmap v61: 33 pgs: 33 active+clean; 578 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 424 B/s rd, 1.1 KiB/s wr, 1 op/s 2024-06-01T01:57:34.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:34 smithi161 ceph-mon[26055]: pgmap v61: 33 pgs: 33 active+clean; 578 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 424 B/s rd, 1.1 KiB/s wr, 1 op/s 2024-06-01T01:57:34.446 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:57:34 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: ::ffff:172.21.15.161 - - [01/Jun/2024:01:57:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-06-01T01:57:36.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:36 smithi116 ceph-mon[29958]: pgmap v62: 33 pgs: 33 active+clean; 579 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 673 B/s rd, 769 B/s wr, 1 op/s 2024-06-01T01:57:36.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:36 smithi116 ceph-mon[29958]: osdmap e63: 8 total, 8 up, 8 in 2024-06-01T01:57:36.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:36 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1745841302' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]: dispatch 2024-06-01T01:57:36.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:36 smithi116 ceph-mon[25082]: pgmap v62: 33 pgs: 33 active+clean; 579 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 673 B/s rd, 769 B/s wr, 1 op/s 2024-06-01T01:57:36.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:36 smithi116 ceph-mon[25082]: osdmap e63: 8 total, 8 up, 8 in 2024-06-01T01:57:36.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:36 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1745841302' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]: dispatch 2024-06-01T01:57:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:36 smithi161 ceph-mon[26055]: pgmap v62: 33 pgs: 33 active+clean; 579 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 673 B/s rd, 769 B/s wr, 1 op/s 2024-06-01T01:57:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:36 smithi161 ceph-mon[26055]: osdmap e63: 8 total, 8 up, 8 in 2024-06-01T01:57:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:36 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1745841302' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]: dispatch 2024-06-01T01:57:37.521 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:37 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1745841302' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]': finished 2024-06-01T01:57:37.521 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:37 smithi161 ceph-mon[26055]: osdmap e64: 8 total, 8 up, 8 in 2024-06-01T01:57:37.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:37 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1745841302' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]': finished 2024-06-01T01:57:37.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:37 smithi116 ceph-mon[25082]: osdmap e64: 8 total, 8 up, 8 in 2024-06-01T01:57:37.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:37 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1745841302' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]': finished 2024-06-01T01:57:37.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:37 smithi116 ceph-mon[29958]: osdmap e64: 8 total, 8 up, 8 in 2024-06-01T01:57:38.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:38 smithi116 ceph-mon[25082]: pgmap v65: 65 pgs: 32 unknown, 33 active+clean; 579 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 1023 B/s wr, 2 op/s 2024-06-01T01:57:38.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:38 smithi116 ceph-mon[25082]: osdmap e65: 8 total, 8 up, 8 in 2024-06-01T01:57:38.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:38 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1745841302' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]: dispatch 2024-06-01T01:57:38.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:38 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:38.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:38 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:38.295 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:57:38 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:57:38] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:57:38.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:38 smithi116 ceph-mon[29958]: pgmap v65: 65 pgs: 32 unknown, 33 active+clean; 579 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 1023 B/s wr, 2 op/s 2024-06-01T01:57:38.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:38 smithi116 ceph-mon[29958]: osdmap e65: 8 total, 8 up, 8 in 2024-06-01T01:57:38.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:38 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1745841302' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]: dispatch 2024-06-01T01:57:38.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:38 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:38.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:38 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:38.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:38 smithi161 ceph-mon[26055]: pgmap v65: 65 pgs: 32 unknown, 33 active+clean; 579 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 1023 B/s wr, 2 op/s 2024-06-01T01:57:38.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:38 smithi161 ceph-mon[26055]: osdmap e65: 8 total, 8 up, 8 in 2024-06-01T01:57:38.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:38 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1745841302' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]: dispatch 2024-06-01T01:57:38.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:38 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:38.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:38 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:38.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:57:38 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ::ffff:172.21.15.161 - - [01/Jun/2024:01:57:38] "GET /metrics HTTP/1.1" 200 192901 "" "Prometheus/2.33.4" 2024-06-01T01:57:39.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:39 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1745841302' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]': finished 2024-06-01T01:57:39.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:39 smithi116 ceph-mon[29958]: osdmap e66: 8 total, 8 up, 8 in 2024-06-01T01:57:39.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:39 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1745841302' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]': finished 2024-06-01T01:57:39.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:39 smithi116 ceph-mon[25082]: osdmap e66: 8 total, 8 up, 8 in 2024-06-01T01:57:39.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:39 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1745841302' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]': finished 2024-06-01T01:57:39.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:39 smithi161 ceph-mon[26055]: osdmap e66: 8 total, 8 up, 8 in 2024-06-01T01:57:40.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:40 smithi116 ceph-mon[25082]: pgmap v68: 97 pgs: 56 unknown, 41 active+clean; 579 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-01T01:57:40.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:40 smithi116 ceph-mon[25082]: osdmap e67: 8 total, 8 up, 8 in 2024-06-01T01:57:40.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:40 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1149639926' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2024-06-01T01:57:40.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:40 smithi116 ceph-mon[25082]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2024-06-01T01:57:40.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:40 smithi116 ceph-mon[29958]: pgmap v68: 97 pgs: 56 unknown, 41 active+clean; 579 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-01T01:57:40.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:40 smithi116 ceph-mon[29958]: osdmap e67: 8 total, 8 up, 8 in 2024-06-01T01:57:40.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:40 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1149639926' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2024-06-01T01:57:40.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:40 smithi116 ceph-mon[29958]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2024-06-01T01:57:40.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:40 smithi161 ceph-mon[26055]: pgmap v68: 97 pgs: 56 unknown, 41 active+clean; 579 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-01T01:57:40.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:40 smithi161 ceph-mon[26055]: osdmap e67: 8 total, 8 up, 8 in 2024-06-01T01:57:40.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:40 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1149639926' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2024-06-01T01:57:40.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:40 smithi161 ceph-mon[26055]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2024-06-01T01:57:41.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:41 smithi116 ceph-mon[25082]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]': finished 2024-06-01T01:57:41.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:41 smithi116 ceph-mon[25082]: osdmap e68: 8 total, 8 up, 8 in 2024-06-01T01:57:41.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:41 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1149639926' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-01T01:57:41.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:41 smithi116 ceph-mon[25082]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-01T01:57:41.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:41 smithi116 ceph-mon[29958]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]': finished 2024-06-01T01:57:41.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:41 smithi116 ceph-mon[29958]: osdmap e68: 8 total, 8 up, 8 in 2024-06-01T01:57:41.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:41 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1149639926' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-01T01:57:41.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:41 smithi116 ceph-mon[29958]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-01T01:57:41.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:41 smithi161 ceph-mon[26055]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]': finished 2024-06-01T01:57:41.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:41 smithi161 ceph-mon[26055]: osdmap e68: 8 total, 8 up, 8 in 2024-06-01T01:57:41.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:41 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1149639926' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-01T01:57:41.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:41 smithi161 ceph-mon[26055]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-01T01:57:42.299 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T01:57:42.299 INFO:teuthology.orchestra.run.smithi116.stdout: "id": "732f953c-d911-4f35-b308-2c89263fc8bc", 2024-06-01T01:57:42.299 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 1, 2024-06-01T01:57:42.299 INFO:teuthology.orchestra.run.smithi116.stdout: "predecessor_uuid": "1c7bc5b8-908b-4920-9391-420ae5b79983", 2024-06-01T01:57:42.299 INFO:teuthology.orchestra.run.smithi116.stdout: "sync_status": [], 2024-06-01T01:57:42.300 INFO:teuthology.orchestra.run.smithi116.stdout: "period_map": { 2024-06-01T01:57:42.300 INFO:teuthology.orchestra.run.smithi116.stdout: "id": "732f953c-d911-4f35-b308-2c89263fc8bc", 2024-06-01T01:57:42.300 INFO:teuthology.orchestra.run.smithi116.stdout: "zonegroups": [ 2024-06-01T01:57:42.300 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-06-01T01:57:42.300 INFO:teuthology.orchestra.run.smithi116.stdout: "id": "027b600c-7676-481b-b1c6-6b1a5a28256b", 2024-06-01T01:57:42.300 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "default", 2024-06-01T01:57:42.300 INFO:teuthology.orchestra.run.smithi116.stdout: "api_name": "default", 2024-06-01T01:57:42.300 INFO:teuthology.orchestra.run.smithi116.stdout: "is_master": "true", 2024-06-01T01:57:42.300 INFO:teuthology.orchestra.run.smithi116.stdout: "endpoints": [], 2024-06-01T01:57:42.300 INFO:teuthology.orchestra.run.smithi116.stdout: "hostnames": [], 2024-06-01T01:57:42.300 INFO:teuthology.orchestra.run.smithi116.stdout: "hostnames_s3website": [], 2024-06-01T01:57:42.300 INFO:teuthology.orchestra.run.smithi116.stdout: "master_zone": "fb01931c-2a09-4e4a-814f-6f0cc6362429", 2024-06-01T01:57:42.301 INFO:teuthology.orchestra.run.smithi116.stdout: "zones": [ 2024-06-01T01:57:42.301 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-06-01T01:57:42.301 INFO:teuthology.orchestra.run.smithi116.stdout: "id": "fb01931c-2a09-4e4a-814f-6f0cc6362429", 2024-06-01T01:57:42.301 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "z", 2024-06-01T01:57:42.301 INFO:teuthology.orchestra.run.smithi116.stdout: "endpoints": [], 2024-06-01T01:57:42.301 INFO:teuthology.orchestra.run.smithi116.stdout: "log_meta": "false", 2024-06-01T01:57:42.301 INFO:teuthology.orchestra.run.smithi116.stdout: "log_data": "false", 2024-06-01T01:57:42.301 INFO:teuthology.orchestra.run.smithi116.stdout: "bucket_index_max_shards": 11, 2024-06-01T01:57:42.301 INFO:teuthology.orchestra.run.smithi116.stdout: "read_only": "false", 2024-06-01T01:57:42.301 INFO:teuthology.orchestra.run.smithi116.stdout: "tier_type": "", 2024-06-01T01:57:42.301 INFO:teuthology.orchestra.run.smithi116.stdout: "sync_from_all": "true", 2024-06-01T01:57:42.301 INFO:teuthology.orchestra.run.smithi116.stdout: "sync_from": [], 2024-06-01T01:57:42.302 INFO:teuthology.orchestra.run.smithi116.stdout: "redirect_zone": "" 2024-06-01T01:57:42.302 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T01:57:42.302 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-06-01T01:57:42.302 INFO:teuthology.orchestra.run.smithi116.stdout: "placement_targets": [ 2024-06-01T01:57:42.302 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-06-01T01:57:42.302 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "default-placement", 2024-06-01T01:57:42.302 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-06-01T01:57:42.302 INFO:teuthology.orchestra.run.smithi116.stdout: "storage_classes": [ 2024-06-01T01:57:42.302 INFO:teuthology.orchestra.run.smithi116.stdout: "STANDARD" 2024-06-01T01:57:42.302 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2024-06-01T01:57:42.302 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T01:57:42.303 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-06-01T01:57:42.303 INFO:teuthology.orchestra.run.smithi116.stdout: "default_placement": "default-placement", 2024-06-01T01:57:42.303 INFO:teuthology.orchestra.run.smithi116.stdout: "realm_id": "1ebaea68-0bcb-4c99-839c-d3107430bc58", 2024-06-01T01:57:42.303 INFO:teuthology.orchestra.run.smithi116.stdout: "sync_policy": { 2024-06-01T01:57:42.303 INFO:teuthology.orchestra.run.smithi116.stdout: "groups": [] 2024-06-01T01:57:42.303 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T01:57:42.303 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T01:57:42.303 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-06-01T01:57:42.303 INFO:teuthology.orchestra.run.smithi116.stdout: "short_zone_ids": [ 2024-06-01T01:57:42.303 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-06-01T01:57:42.303 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "fb01931c-2a09-4e4a-814f-6f0cc6362429", 2024-06-01T01:57:42.303 INFO:teuthology.orchestra.run.smithi116.stdout: "val": 542099358 2024-06-01T01:57:42.303 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T01:57:42.304 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2024-06-01T01:57:42.304 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T01:57:42.304 INFO:teuthology.orchestra.run.smithi116.stdout: "master_zonegroup": "027b600c-7676-481b-b1c6-6b1a5a28256b", 2024-06-01T01:57:42.304 INFO:teuthology.orchestra.run.smithi116.stdout: "master_zone": "fb01931c-2a09-4e4a-814f-6f0cc6362429", 2024-06-01T01:57:42.304 INFO:teuthology.orchestra.run.smithi116.stdout: "period_config": { 2024-06-01T01:57:42.304 INFO:teuthology.orchestra.run.smithi116.stdout: "bucket_quota": { 2024-06-01T01:57:42.304 INFO:teuthology.orchestra.run.smithi116.stdout: "enabled": false, 2024-06-01T01:57:42.304 INFO:teuthology.orchestra.run.smithi116.stdout: "check_on_raw": false, 2024-06-01T01:57:42.304 INFO:teuthology.orchestra.run.smithi116.stdout: "max_size": -1, 2024-06-01T01:57:42.304 INFO:teuthology.orchestra.run.smithi116.stdout: "max_size_kb": 0, 2024-06-01T01:57:42.304 INFO:teuthology.orchestra.run.smithi116.stdout: "max_objects": -1 2024-06-01T01:57:42.304 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T01:57:42.304 INFO:teuthology.orchestra.run.smithi116.stdout: "user_quota": { 2024-06-01T01:57:42.304 INFO:teuthology.orchestra.run.smithi116.stdout: "enabled": false, 2024-06-01T01:57:42.305 INFO:teuthology.orchestra.run.smithi116.stdout: "check_on_raw": false, 2024-06-01T01:57:42.305 INFO:teuthology.orchestra.run.smithi116.stdout: "max_size": -1, 2024-06-01T01:57:42.305 INFO:teuthology.orchestra.run.smithi116.stdout: "max_size_kb": 0, 2024-06-01T01:57:42.305 INFO:teuthology.orchestra.run.smithi116.stdout: "max_objects": -1 2024-06-01T01:57:42.305 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T01:57:42.305 INFO:teuthology.orchestra.run.smithi116.stdout: "user_ratelimit": { 2024-06-01T01:57:42.305 INFO:teuthology.orchestra.run.smithi116.stdout: "max_read_ops": 0, 2024-06-01T01:57:42.305 INFO:teuthology.orchestra.run.smithi116.stdout: "max_write_ops": 0, 2024-06-01T01:57:42.305 INFO:teuthology.orchestra.run.smithi116.stdout: "max_read_bytes": 0, 2024-06-01T01:57:42.305 INFO:teuthology.orchestra.run.smithi116.stdout: "max_write_bytes": 0, 2024-06-01T01:57:42.305 INFO:teuthology.orchestra.run.smithi116.stdout: "enabled": false 2024-06-01T01:57:42.305 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T01:57:42.305 INFO:teuthology.orchestra.run.smithi116.stdout: "bucket_ratelimit": { 2024-06-01T01:57:42.305 INFO:teuthology.orchestra.run.smithi116.stdout: "max_read_ops": 0, 2024-06-01T01:57:42.305 INFO:teuthology.orchestra.run.smithi116.stdout: "max_write_ops": 0, 2024-06-01T01:57:42.305 INFO:teuthology.orchestra.run.smithi116.stdout: "max_read_bytes": 0, 2024-06-01T01:57:42.306 INFO:teuthology.orchestra.run.smithi116.stdout: "max_write_bytes": 0, 2024-06-01T01:57:42.306 INFO:teuthology.orchestra.run.smithi116.stdout: "enabled": false 2024-06-01T01:57:42.306 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T01:57:42.306 INFO:teuthology.orchestra.run.smithi116.stdout: "anonymous_ratelimit": { 2024-06-01T01:57:42.306 INFO:teuthology.orchestra.run.smithi116.stdout: "max_read_ops": 0, 2024-06-01T01:57:42.306 INFO:teuthology.orchestra.run.smithi116.stdout: "max_write_ops": 0, 2024-06-01T01:57:42.306 INFO:teuthology.orchestra.run.smithi116.stdout: "max_read_bytes": 0, 2024-06-01T01:57:42.306 INFO:teuthology.orchestra.run.smithi116.stdout: "max_write_bytes": 0, 2024-06-01T01:57:42.306 INFO:teuthology.orchestra.run.smithi116.stdout: "enabled": false 2024-06-01T01:57:42.306 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T01:57:42.306 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T01:57:42.306 INFO:teuthology.orchestra.run.smithi116.stdout: "realm_id": "1ebaea68-0bcb-4c99-839c-d3107430bc58", 2024-06-01T01:57:42.306 INFO:teuthology.orchestra.run.smithi116.stdout: "realm_name": "r", 2024-06-01T01:57:42.307 INFO:teuthology.orchestra.run.smithi116.stdout: "realm_epoch": 2 2024-06-01T01:57:42.307 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T01:57:42.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:42 smithi116 ceph-mon[25082]: pgmap v71: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 50 MiB used, 715 GiB / 715 GiB avail; 6.0 KiB/s rd, 511 B/s wr, 7 op/s 2024-06-01T01:57:42.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:42 smithi116 ceph-mon[25082]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-06-01T01:57:42.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:42 smithi116 ceph-mon[25082]: osdmap e69: 8 total, 8 up, 8 in 2024-06-01T01:57:42.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:42 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1149639926' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]: dispatch 2024-06-01T01:57:42.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:42 smithi116 ceph-mon[25082]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]: dispatch 2024-06-01T01:57:42.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:42 smithi116 ceph-mon[29958]: pgmap v71: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 50 MiB used, 715 GiB / 715 GiB avail; 6.0 KiB/s rd, 511 B/s wr, 7 op/s 2024-06-01T01:57:42.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:42 smithi116 ceph-mon[29958]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-06-01T01:57:42.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:42 smithi116 ceph-mon[29958]: osdmap e69: 8 total, 8 up, 8 in 2024-06-01T01:57:42.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:42 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1149639926' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]: dispatch 2024-06-01T01:57:42.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:42 smithi116 ceph-mon[29958]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]: dispatch 2024-06-01T01:57:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:42 smithi161 ceph-mon[26055]: pgmap v71: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 50 MiB used, 715 GiB / 715 GiB avail; 6.0 KiB/s rd, 511 B/s wr, 7 op/s 2024-06-01T01:57:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:42 smithi161 ceph-mon[26055]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-06-01T01:57:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:42 smithi161 ceph-mon[26055]: osdmap e69: 8 total, 8 up, 8 in 2024-06-01T01:57:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:42 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1149639926' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]: dispatch 2024-06-01T01:57:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:42 smithi161 ceph-mon[26055]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]: dispatch 2024-06-01T01:57:43.075 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph orch apply rgw foo --realm r --zone z --placement=2 --port=8000' 2024-06-01T01:57:43.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:43 smithi116 ceph-mon[25082]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]': finished 2024-06-01T01:57:43.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:43 smithi116 ceph-mon[25082]: osdmap e70: 8 total, 8 up, 8 in 2024-06-01T01:57:43.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T01:57:43.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T01:57:43.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:43 smithi116 ceph-mon[29958]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]': finished 2024-06-01T01:57:43.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:43 smithi116 ceph-mon[29958]: osdmap e70: 8 total, 8 up, 8 in 2024-06-01T01:57:43.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T01:57:43.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T01:57:43.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:43 smithi161 ceph-mon[26055]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]': finished 2024-06-01T01:57:43.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:43 smithi161 ceph-mon[26055]: osdmap e70: 8 total, 8 up, 8 in 2024-06-01T01:57:43.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T01:57:43.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T01:57:44.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:44 smithi116 ceph-mon[25082]: pgmap v74: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 50 MiB used, 715 GiB / 715 GiB avail; 4.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-01T01:57:44.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:44 smithi116 ceph-mon[29958]: pgmap v74: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 50 MiB used, 715 GiB / 715 GiB avail; 4.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-01T01:57:44.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:44 smithi161 ceph-mon[26055]: pgmap v74: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 50 MiB used, 715 GiB / 715 GiB avail; 4.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-01T01:57:44.695 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:57:44 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: ::ffff:172.21.15.161 - - [01/Jun/2024:01:57:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-06-01T01:57:45.294 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:57:45 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:57:45] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:57:46.396 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:46 smithi116 ceph-mon[25082]: pgmap v75: 129 pgs: 129 active+clean; 581 KiB data, 52 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-06-01T01:57:46.397 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:46 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:46.397 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:46 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:46.397 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:46 smithi116 ceph-mon[29958]: pgmap v75: 129 pgs: 129 active+clean; 581 KiB data, 52 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-06-01T01:57:46.397 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:46 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:46.397 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:46 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:46.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:46 smithi161 ceph-mon[26055]: pgmap v75: 129 pgs: 129 active+clean; 581 KiB data, 52 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-06-01T01:57:46.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:46 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:46.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:46 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:46.717 INFO:teuthology.orchestra.run.smithi116.stdout:Scheduled rgw.foo update... 2024-06-01T01:57:47.685 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph orch apply rgw smpl' 2024-06-01T01:57:47.817 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:47 smithi161 ceph-mon[26055]: from='client.24583 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo", "realm": "r", "zone": "z", "placement": "2", "port": 8000, "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:57:47.817 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:47 smithi161 ceph-mon[26055]: Saving service rgw.foo spec with placement count:2 2024-06-01T01:57:47.817 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:47 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:47.817 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:47 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:57:47.817 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:47 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:57:47.817 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:47 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:47.817 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:47 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:47.817 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:47 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:47.818 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:47 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:47.818 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:47 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi116.vzjrxj", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T01:57:47.818 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:47 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi116.vzjrxj", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-06-01T01:57:47.818 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:47 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:47.818 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:47 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:57:47.940 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:47 smithi116 ceph-mon[25082]: from='client.24583 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo", "realm": "r", "zone": "z", "placement": "2", "port": 8000, "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:57:47.940 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:47 smithi116 ceph-mon[25082]: Saving service rgw.foo spec with placement count:2 2024-06-01T01:57:47.941 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:47 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:47.941 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:47 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:57:47.941 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:47 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:57:47.941 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:47 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:47.941 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:47 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:47.942 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:47 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:47.942 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:47 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:47.942 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:47 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi116.vzjrxj", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T01:57:47.942 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:47 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi116.vzjrxj", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-06-01T01:57:47.942 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:47 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:47.942 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:47 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:57:47.943 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:47 smithi116 ceph-mon[29958]: from='client.24583 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo", "realm": "r", "zone": "z", "placement": "2", "port": 8000, "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:57:47.943 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:47 smithi116 ceph-mon[29958]: Saving service rgw.foo spec with placement count:2 2024-06-01T01:57:47.943 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:47 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:47.943 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:47 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:57:47.943 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:47 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:57:47.943 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:47 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:47.943 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:47 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:47.944 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:47 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:47.944 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:47 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:47.944 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:47 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi116.vzjrxj", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T01:57:47.944 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:47 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi116.vzjrxj", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-06-01T01:57:47.944 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:47 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:47.944 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:47 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:57:48.544 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:57:48 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ::ffff:172.21.15.161 - - [01/Jun/2024:01:57:48] "GET /metrics HTTP/1.1" 200 196740 "" "Prometheus/2.33.4" 2024-06-01T01:57:48.942 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:48 smithi116 ceph-mon[29958]: Saving service rgw.foo spec with placement count:2 2024-06-01T01:57:48.943 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:48 smithi116 ceph-mon[29958]: Deploying daemon rgw.foo.smithi116.vzjrxj on smithi116 2024-06-01T01:57:48.943 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:48 smithi116 ceph-mon[29958]: pgmap v76: 129 pgs: 129 active+clean; 581 KiB data, 52 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-06-01T01:57:48.943 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:48 smithi116 ceph-mon[25082]: Saving service rgw.foo spec with placement count:2 2024-06-01T01:57:48.943 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:48 smithi116 ceph-mon[25082]: Deploying daemon rgw.foo.smithi116.vzjrxj on smithi116 2024-06-01T01:57:48.943 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:48 smithi116 ceph-mon[25082]: pgmap v76: 129 pgs: 129 active+clean; 581 KiB data, 52 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-06-01T01:57:49.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:48 smithi161 ceph-mon[26055]: Saving service rgw.foo spec with placement count:2 2024-06-01T01:57:49.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:48 smithi161 ceph-mon[26055]: Deploying daemon rgw.foo.smithi116.vzjrxj on smithi116 2024-06-01T01:57:49.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:48 smithi161 ceph-mon[26055]: pgmap v76: 129 pgs: 129 active+clean; 581 KiB data, 52 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-06-01T01:57:50.992 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:50 smithi116 ceph-mon[25082]: pgmap v77: 129 pgs: 129 active+clean; 581 KiB data, 52 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 895 B/s wr, 2 op/s 2024-06-01T01:57:50.992 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:50 smithi116 ceph-mon[29958]: pgmap v77: 129 pgs: 129 active+clean; 581 KiB data, 52 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 895 B/s wr, 2 op/s 2024-06-01T01:57:51.004 INFO:teuthology.orchestra.run.smithi116.stdout:Scheduled rgw.smpl update... 2024-06-01T01:57:51.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:50 smithi161 ceph-mon[26055]: pgmap v77: 129 pgs: 129 active+clean; 581 KiB data, 52 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 895 B/s wr, 2 op/s 2024-06-01T01:57:52.072 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph osd pool create foo' 2024-06-01T01:57:52.258 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:52 smithi161 ceph-mon[26055]: pgmap v78: 129 pgs: 129 active+clean; 581 KiB data, 52 MiB used, 715 GiB / 715 GiB avail; 752 B/s rd, 752 B/s wr, 1 op/s 2024-06-01T01:57:52.258 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:52 smithi161 ceph-mon[26055]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "smpl", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:57:52.258 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:52 smithi161 ceph-mon[26055]: Saving service rgw.smpl spec with placement count:2 2024-06-01T01:57:52.258 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:52 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:52.259 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:52 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:52.259 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:52 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi161.ashqxz", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T01:57:52.259 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:52 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi161.ashqxz", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-06-01T01:57:52.259 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:52 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:52.259 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:52 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:57:52.259 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:52 smithi161 ceph-mon[26055]: Deploying daemon rgw.foo.smithi161.ashqxz on smithi161 2024-06-01T01:57:52.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:52 smithi116 ceph-mon[25082]: pgmap v78: 129 pgs: 129 active+clean; 581 KiB data, 52 MiB used, 715 GiB / 715 GiB avail; 752 B/s rd, 752 B/s wr, 1 op/s 2024-06-01T01:57:52.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:52 smithi116 ceph-mon[25082]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "smpl", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:57:52.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:52 smithi116 ceph-mon[25082]: Saving service rgw.smpl spec with placement count:2 2024-06-01T01:57:52.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:52 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:52.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:52 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:52.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:52 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi161.ashqxz", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T01:57:52.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:52 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi161.ashqxz", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-06-01T01:57:52.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:52 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:52.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:52 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:57:52.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:52 smithi116 ceph-mon[25082]: Deploying daemon rgw.foo.smithi161.ashqxz on smithi161 2024-06-01T01:57:52.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:52 smithi116 ceph-mon[29958]: pgmap v78: 129 pgs: 129 active+clean; 581 KiB data, 52 MiB used, 715 GiB / 715 GiB avail; 752 B/s rd, 752 B/s wr, 1 op/s 2024-06-01T01:57:52.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:52 smithi116 ceph-mon[29958]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "smpl", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:57:52.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:52 smithi116 ceph-mon[29958]: Saving service rgw.smpl spec with placement count:2 2024-06-01T01:57:52.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:52 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:52.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:52 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:52.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:52 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi161.ashqxz", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T01:57:52.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:52 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi161.ashqxz", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-06-01T01:57:52.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:52 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:52.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:52 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:57:52.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:52 smithi116 ceph-mon[29958]: Deploying daemon rgw.foo.smithi161.ashqxz on smithi161 2024-06-01T01:57:54.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:54 smithi116 ceph-mon[29958]: pgmap v79: 129 pgs: 129 active+clean; 581 KiB data, 52 MiB used, 715 GiB / 715 GiB avail; 680 B/s rd, 680 B/s wr, 1 op/s 2024-06-01T01:57:54.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:54 smithi116 ceph-mon[25082]: pgmap v79: 129 pgs: 129 active+clean; 581 KiB data, 52 MiB used, 715 GiB / 715 GiB avail; 680 B/s rd, 680 B/s wr, 1 op/s 2024-06-01T01:57:54.307 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:54 smithi161 ceph-mon[26055]: pgmap v79: 129 pgs: 129 active+clean; 581 KiB data, 52 MiB used, 715 GiB / 715 GiB avail; 680 B/s rd, 680 B/s wr, 1 op/s 2024-06-01T01:57:54.695 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:57:54 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: ::ffff:172.21.15.161 - - [01/Jun/2024:01:57:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-06-01T01:57:55.023 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'foo' created 2024-06-01T01:57:55.159 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:55 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/229983523' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo"}]: dispatch 2024-06-01T01:57:55.159 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:55 smithi161 ceph-mon[26055]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo"}]: dispatch 2024-06-01T01:57:55.159 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:55 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:55.159 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:55 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:57:55.159 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:55 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:57:55.159 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:55 smithi161 ceph-mon[26055]: Checking dashboard <-> RGW credentials 2024-06-01T01:57:55.159 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:55 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:55.159 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:55 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:55.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:55 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:55.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:55 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:57:55.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:55 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:57:55.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:55 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:57:55.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:55 smithi161 ceph-mon[26055]: Checking dashboard <-> RGW credentials 2024-06-01T01:57:55.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:55 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:55.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:55 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/229983523' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo"}]: dispatch 2024-06-01T01:57:55.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:55 smithi116 ceph-mon[25082]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo"}]: dispatch 2024-06-01T01:57:55.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:55 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:55.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:55 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:57:55.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:55 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:57:55.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:55 smithi116 ceph-mon[25082]: Checking dashboard <-> RGW credentials 2024-06-01T01:57:55.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:55 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:55.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:55 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:55.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:55 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:55.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:55 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:57:55.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:55 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:57:55.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:55 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:57:55.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:55 smithi116 ceph-mon[25082]: Checking dashboard <-> RGW credentials 2024-06-01T01:57:55.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:55 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:55.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:55 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/229983523' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo"}]: dispatch 2024-06-01T01:57:55.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:55 smithi116 ceph-mon[29958]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo"}]: dispatch 2024-06-01T01:57:55.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:55 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:55.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:55 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:57:55.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:55 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:57:55.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:55 smithi116 ceph-mon[29958]: Checking dashboard <-> RGW credentials 2024-06-01T01:57:55.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:55 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:55.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:55 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:55.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:55 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:55.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:55 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:57:55.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:55 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:57:55.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:55 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:57:55.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:55 smithi116 ceph-mon[29958]: Checking dashboard <-> RGW credentials 2024-06-01T01:57:55.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:55 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:55.740 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'rbd pool init foo' 2024-06-01T01:57:56.183 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:56 smithi161 ceph-mon[26055]: pgmap v80: 129 pgs: 129 active+clean; 584 KiB data, 54 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s rd, 3.3 KiB/s wr, 129 op/s 2024-06-01T01:57:56.183 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:56 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:57:56.183 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:56 smithi161 ceph-mon[26055]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "foo"}]': finished 2024-06-01T01:57:56.183 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:56 smithi161 ceph-mon[26055]: osdmap e71: 8 total, 8 up, 8 in 2024-06-01T01:57:56.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:56 smithi116 ceph-mon[25082]: pgmap v80: 129 pgs: 129 active+clean; 584 KiB data, 54 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s rd, 3.3 KiB/s wr, 129 op/s 2024-06-01T01:57:56.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:56 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:57:56.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:56 smithi116 ceph-mon[25082]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "foo"}]': finished 2024-06-01T01:57:56.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:56 smithi116 ceph-mon[25082]: osdmap e71: 8 total, 8 up, 8 in 2024-06-01T01:57:56.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:56 smithi116 ceph-mon[29958]: pgmap v80: 129 pgs: 129 active+clean; 584 KiB data, 54 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s rd, 3.3 KiB/s wr, 129 op/s 2024-06-01T01:57:56.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:56 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:57:56.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:56 smithi116 ceph-mon[29958]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "foo"}]': finished 2024-06-01T01:57:56.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:56 smithi116 ceph-mon[29958]: osdmap e71: 8 total, 8 up, 8 in 2024-06-01T01:57:57.113 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:57 smithi116 ceph-mon[25082]: osdmap e72: 8 total, 8 up, 8 in 2024-06-01T01:57:57.113 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:57 smithi116 ceph-mon[29958]: osdmap e72: 8 total, 8 up, 8 in 2024-06-01T01:57:57.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:57 smithi161 ceph-mon[26055]: osdmap e72: 8 total, 8 up, 8 in 2024-06-01T01:57:58.281 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:58 smithi116 ceph-mon[25082]: pgmap v83: 161 pgs: 32 unknown, 129 active+clean; 584 KiB data, 54 MiB used, 715 GiB / 715 GiB avail; 89 KiB/s rd, 4.1 KiB/s wr, 191 op/s 2024-06-01T01:57:58.281 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:58 smithi116 ceph-mon[25082]: osdmap e73: 8 total, 8 up, 8 in 2024-06-01T01:57:58.281 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:58 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:58.281 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:57:58 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:57:58] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:57:58.282 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:58 smithi116 ceph-mon[29958]: pgmap v83: 161 pgs: 32 unknown, 129 active+clean; 584 KiB data, 54 MiB used, 715 GiB / 715 GiB avail; 89 KiB/s rd, 4.1 KiB/s wr, 191 op/s 2024-06-01T01:57:58.282 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:58 smithi116 ceph-mon[29958]: osdmap e73: 8 total, 8 up, 8 in 2024-06-01T01:57:58.282 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:58 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:58.319 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:58 smithi161 ceph-mon[26055]: pgmap v83: 161 pgs: 32 unknown, 129 active+clean; 584 KiB data, 54 MiB used, 715 GiB / 715 GiB avail; 89 KiB/s rd, 4.1 KiB/s wr, 191 op/s 2024-06-01T01:57:58.319 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:58 smithi161 ceph-mon[26055]: osdmap e73: 8 total, 8 up, 8 in 2024-06-01T01:57:58.319 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:58 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:58.544 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:57:58 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ::ffff:172.21.15.161 - - [01/Jun/2024:01:57:58] "GET /metrics HTTP/1.1" 200 207164 "" "Prometheus/2.33.4" 2024-06-01T01:57:59.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:59 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:59.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:57:59 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/3744668691' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]: dispatch 2024-06-01T01:57:59.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:59 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:59.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:57:59 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/3744668691' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]: dispatch 2024-06-01T01:57:59.544 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:57:59 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:57:59] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:57:59.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:59 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:57:59.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:57:59 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/3744668691' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]: dispatch 2024-06-01T01:58:00.134 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:00 smithi116 ceph-mon[29958]: pgmap v85: 161 pgs: 19 unknown, 142 active+clean; 584 KiB data, 59 MiB used, 715 GiB / 715 GiB avail; 208 KiB/s rd, 5.7 KiB/s wr, 387 op/s 2024-06-01T01:58:00.134 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:00 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/3744668691' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]': finished 2024-06-01T01:58:00.134 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:00 smithi116 ceph-mon[29958]: osdmap e74: 8 total, 8 up, 8 in 2024-06-01T01:58:00.134 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:00 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:00.134 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:00 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:00.134 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:00 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:00.134 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:00 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:00.135 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:00 smithi116 ceph-mon[25082]: pgmap v85: 161 pgs: 19 unknown, 142 active+clean; 584 KiB data, 59 MiB used, 715 GiB / 715 GiB avail; 208 KiB/s rd, 5.7 KiB/s wr, 387 op/s 2024-06-01T01:58:00.135 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:00 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/3744668691' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]': finished 2024-06-01T01:58:00.135 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:00 smithi116 ceph-mon[25082]: osdmap e74: 8 total, 8 up, 8 in 2024-06-01T01:58:00.135 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:00 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:00.135 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:00 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:00.135 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:00 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:00.135 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:00 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:00.135 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:57:59 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:57:59] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:58:00.136 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:58:00 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:58:00] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:58:00.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:00 smithi161 ceph-mon[26055]: pgmap v85: 161 pgs: 19 unknown, 142 active+clean; 584 KiB data, 59 MiB used, 715 GiB / 715 GiB avail; 208 KiB/s rd, 5.7 KiB/s wr, 387 op/s 2024-06-01T01:58:00.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:00 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/3744668691' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]': finished 2024-06-01T01:58:00.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:00 smithi161 ceph-mon[26055]: osdmap e74: 8 total, 8 up, 8 in 2024-06-01T01:58:00.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:00 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:00.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:00 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:00.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:00 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:00.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:00 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:00.452 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:58:00 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:58:00] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:58:01.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:01 smithi161 ceph-mon[26055]: osdmap e75: 8 total, 8 up, 8 in 2024-06-01T01:58:01.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:01 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:01.495 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:01 smithi116 ceph-mon[25082]: osdmap e75: 8 total, 8 up, 8 in 2024-06-01T01:58:01.495 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:01 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:01.495 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:01 smithi116 ceph-mon[29958]: osdmap e75: 8 total, 8 up, 8 in 2024-06-01T01:58:01.495 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:01 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:01.837 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph orch apply iscsi foo u p' 2024-06-01T01:58:02.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:02 smithi161 ceph-mon[26055]: pgmap v88: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 223 KiB/s rd, 646 B/s wr, 334 op/s 2024-06-01T01:58:02.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:02 smithi161 ceph-mon[26055]: osdmap e76: 8 total, 8 up, 8 in 2024-06-01T01:58:02.449 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:02 smithi116 ceph-mon[29958]: pgmap v88: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 223 KiB/s rd, 646 B/s wr, 334 op/s 2024-06-01T01:58:02.449 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:02 smithi116 ceph-mon[29958]: osdmap e76: 8 total, 8 up, 8 in 2024-06-01T01:58:02.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:02 smithi116 ceph-mon[25082]: pgmap v88: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 223 KiB/s rd, 646 B/s wr, 334 op/s 2024-06-01T01:58:02.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:02 smithi116 ceph-mon[25082]: osdmap e76: 8 total, 8 up, 8 in 2024-06-01T01:58:03.537 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:58:03 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:58:03] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:58:04.533 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:04 smithi116 ceph-mon[29958]: pgmap v90: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 185 KiB/s rd, 534 B/s wr, 276 op/s 2024-06-01T01:58:04.534 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:04 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:04.534 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:04 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:04.534 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:04 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:58:04.534 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:04 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:58:04.534 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:04 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:04.534 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:04 smithi116 ceph-mon[29958]: Saving service rgw.smpl spec with placement count:2 2024-06-01T01:58:04.534 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:04 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:04.534 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:04 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi116.hokbmf", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T01:58:04.534 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:04 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi116.hokbmf", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-06-01T01:58:04.534 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:04 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:04.534 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:04 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:58:04.534 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:04 smithi116 ceph-mon[29958]: Deploying daemon rgw.smpl.smithi116.hokbmf on smithi116 2024-06-01T01:58:04.535 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:04 smithi116 ceph-mon[25082]: pgmap v90: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 185 KiB/s rd, 534 B/s wr, 276 op/s 2024-06-01T01:58:04.535 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:04 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:04.535 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:04 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:04.535 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:04 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:58:04.535 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:04 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:58:04.535 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:04 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:04.535 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:04 smithi116 ceph-mon[25082]: Saving service rgw.smpl spec with placement count:2 2024-06-01T01:58:04.535 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:04 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:04.535 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:04 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi116.hokbmf", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T01:58:04.535 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:04 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi116.hokbmf", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-06-01T01:58:04.535 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:04 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:04.536 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:04 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:58:04.536 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:04 smithi116 ceph-mon[25082]: Deploying daemon rgw.smpl.smithi116.hokbmf on smithi116 2024-06-01T01:58:04.695 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:58:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: ::ffff:172.21.15.161 - - [01/Jun/2024:01:58:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-06-01T01:58:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:04 smithi161 ceph-mon[26055]: pgmap v90: 161 pgs: 161 active+clean; 585 KiB data, 68 MiB used, 715 GiB / 715 GiB avail; 185 KiB/s rd, 534 B/s wr, 276 op/s 2024-06-01T01:58:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:04 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:04 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:04 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:58:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:04 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:58:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:04 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:04 smithi161 ceph-mon[26055]: Saving service rgw.smpl spec with placement count:2 2024-06-01T01:58:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:04 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:04 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi116.hokbmf", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T01:58:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:04 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi116.hokbmf", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-06-01T01:58:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:04 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:04.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:04 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:58:04.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:04 smithi161 ceph-mon[26055]: Deploying daemon rgw.smpl.smithi116.hokbmf on smithi116 2024-06-01T01:58:04.996 INFO:teuthology.orchestra.run.smithi116.stdout:Scheduled iscsi.foo update... 2024-06-01T01:58:06.129 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:06 smithi116 ceph-mon[29958]: pgmap v91: 161 pgs: 161 active+clean; 585 KiB data, 69 MiB used, 715 GiB / 715 GiB avail; 102 KiB/s rd, 682 B/s wr, 155 op/s 2024-06-01T01:58:06.129 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:06 smithi116 ceph-mon[29958]: from='client.24721 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "foo", "api_user": "u", "api_password": "p", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:58:06.129 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:06 smithi116 ceph-mon[29958]: Saving service iscsi.foo spec with placement count:1 2024-06-01T01:58:06.129 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:06 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:06.129 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:05 smithi116 ceph-mon[25082]: pgmap v91: 161 pgs: 161 active+clean; 585 KiB data, 69 MiB used, 715 GiB / 715 GiB avail; 102 KiB/s rd, 682 B/s wr, 155 op/s 2024-06-01T01:58:06.129 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:05 smithi116 ceph-mon[25082]: from='client.24721 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "foo", "api_user": "u", "api_password": "p", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:58:06.130 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:05 smithi116 ceph-mon[25082]: Saving service iscsi.foo spec with placement count:1 2024-06-01T01:58:06.130 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:05 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:06.156 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'sleep 120' 2024-06-01T01:58:06.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:05 smithi161 ceph-mon[26055]: pgmap v91: 161 pgs: 161 active+clean; 585 KiB data, 69 MiB used, 715 GiB / 715 GiB avail; 102 KiB/s rd, 682 B/s wr, 155 op/s 2024-06-01T01:58:06.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:05 smithi161 ceph-mon[26055]: from='client.24721 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "foo", "api_user": "u", "api_password": "p", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T01:58:06.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:05 smithi161 ceph-mon[26055]: Saving service iscsi.foo spec with placement count:1 2024-06-01T01:58:06.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:05 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:08.144 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:08 smithi116 ceph-mon[25082]: pgmap v92: 161 pgs: 161 active+clean; 585 KiB data, 69 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 267 B/s wr, 17 op/s 2024-06-01T01:58:08.145 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:08 smithi116 ceph-mon[29958]: pgmap v92: 161 pgs: 161 active+clean; 585 KiB data, 69 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 267 B/s wr, 17 op/s 2024-06-01T01:58:08.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:08 smithi161 ceph-mon[26055]: pgmap v92: 161 pgs: 161 active+clean; 585 KiB data, 69 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 267 B/s wr, 17 op/s 2024-06-01T01:58:08.787 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:58:08 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ::ffff:172.21.15.161 - - [01/Jun/2024:01:58:08] "GET /metrics HTTP/1.1" 200 207164 "" "Prometheus/2.33.4" 2024-06-01T01:58:10.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:09 smithi161 ceph-mon[26055]: pgmap v93: 161 pgs: 161 active+clean; 585 KiB data, 69 MiB used, 715 GiB / 715 GiB avail; 13 KiB/s rd, 236 B/s wr, 20 op/s 2024-06-01T01:58:10.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:09 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:10.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:09 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi161.axjlzw", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T01:58:10.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:09 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi161.axjlzw", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-06-01T01:58:10.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:09 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:10.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:09 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:58:10.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:09 smithi161 ceph-mon[26055]: Deploying daemon rgw.smpl.smithi161.axjlzw on smithi161 2024-06-01T01:58:10.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:09 smithi161 ceph-mon[26055]: osdmap e77: 8 total, 8 up, 8 in 2024-06-01T01:58:10.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:09 smithi116 ceph-mon[25082]: pgmap v93: 161 pgs: 161 active+clean; 585 KiB data, 69 MiB used, 715 GiB / 715 GiB avail; 13 KiB/s rd, 236 B/s wr, 20 op/s 2024-06-01T01:58:10.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:09 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:10.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:09 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi161.axjlzw", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T01:58:10.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:09 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi161.axjlzw", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-06-01T01:58:10.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:09 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:10.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:09 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:58:10.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:09 smithi116 ceph-mon[25082]: Deploying daemon rgw.smpl.smithi161.axjlzw on smithi161 2024-06-01T01:58:10.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:09 smithi116 ceph-mon[25082]: osdmap e77: 8 total, 8 up, 8 in 2024-06-01T01:58:10.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:09 smithi116 ceph-mon[29958]: pgmap v93: 161 pgs: 161 active+clean; 585 KiB data, 69 MiB used, 715 GiB / 715 GiB avail; 13 KiB/s rd, 236 B/s wr, 20 op/s 2024-06-01T01:58:10.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:09 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:10.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:09 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi161.axjlzw", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T01:58:10.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:09 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi161.axjlzw", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-06-01T01:58:10.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:09 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:10.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:09 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:58:10.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:09 smithi116 ceph-mon[29958]: Deploying daemon rgw.smpl.smithi161.axjlzw on smithi161 2024-06-01T01:58:10.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:09 smithi116 ceph-mon[29958]: osdmap e77: 8 total, 8 up, 8 in 2024-06-01T01:58:12.567 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:12 smithi161 ceph-mon[26055]: pgmap v95: 161 pgs: 161 active+clean; 585 KiB data, 69 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 212 B/s wr, 23 op/s 2024-06-01T01:58:12.567 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:12 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:12.567 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:12 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:58:12.567 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:12 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:58:12.567 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:12 smithi161 ceph-mon[26055]: Checking dashboard <-> RGW credentials 2024-06-01T01:58:12.567 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:12 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:12.567 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:12 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:58:12.567 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:12 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:58:12.567 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:12 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:58:12.567 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:12 smithi161 ceph-mon[26055]: Checking dashboard <-> RGW credentials 2024-06-01T01:58:12.567 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:12 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:12.581 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:12 smithi116 ceph-mon[29958]: pgmap v95: 161 pgs: 161 active+clean; 585 KiB data, 69 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 212 B/s wr, 23 op/s 2024-06-01T01:58:12.581 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:12 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:12.581 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:12 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:58:12.581 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:12 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:58:12.581 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:12 smithi116 ceph-mon[29958]: Checking dashboard <-> RGW credentials 2024-06-01T01:58:12.581 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:12 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:12.581 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:12 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:58:12.582 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:12 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:58:12.582 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:12 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:58:12.582 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:12 smithi116 ceph-mon[29958]: Checking dashboard <-> RGW credentials 2024-06-01T01:58:12.582 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:12 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:12.582 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:12 smithi116 ceph-mon[25082]: pgmap v95: 161 pgs: 161 active+clean; 585 KiB data, 69 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 212 B/s wr, 23 op/s 2024-06-01T01:58:12.582 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:12 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:12.582 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:12 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:58:12.582 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:12 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:58:12.583 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:12 smithi116 ceph-mon[25082]: Checking dashboard <-> RGW credentials 2024-06-01T01:58:12.583 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:12 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:12.583 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:12 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:58:12.583 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:12 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:58:12.583 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:12 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:58:12.583 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:12 smithi116 ceph-mon[25082]: Checking dashboard <-> RGW credentials 2024-06-01T01:58:12.583 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:12 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:13.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:13 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:58:13.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:13 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:13.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:13 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:58:13.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:13 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:13.552 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:13 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:58:13.552 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:13 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:14.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:14 smithi161 ceph-mon[26055]: pgmap v96: 161 pgs: 161 active+clean; 585 KiB data, 69 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 204 B/s wr, 22 op/s 2024-06-01T01:58:14.696 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:58:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: ::ffff:172.21.15.161 - - [01/Jun/2024:01:58:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-06-01T01:58:14.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:14 smithi116 ceph-mon[25082]: pgmap v96: 161 pgs: 161 active+clean; 585 KiB data, 69 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 204 B/s wr, 22 op/s 2024-06-01T01:58:14.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:14 smithi116 ceph-mon[29958]: pgmap v96: 161 pgs: 161 active+clean; 585 KiB data, 69 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 204 B/s wr, 22 op/s 2024-06-01T01:58:15.544 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:58:15 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:58:15] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:58:15.976 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:58:15 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:58:15] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:58:16.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:16 smithi116 ceph-mon[25082]: pgmap v97: 161 pgs: 161 active+clean; 585 KiB data, 93 MiB used, 715 GiB / 715 GiB avail; 195 KiB/s rd, 204 B/s wr, 296 op/s 2024-06-01T01:58:16.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:16 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:16.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:16 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:16.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:16 smithi116 ceph-mon[29958]: pgmap v97: 161 pgs: 161 active+clean; 585 KiB data, 93 MiB used, 715 GiB / 715 GiB avail; 195 KiB/s rd, 204 B/s wr, 296 op/s 2024-06-01T01:58:16.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:16 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:16.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:16 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:16.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:16 smithi161 ceph-mon[26055]: pgmap v97: 161 pgs: 161 active+clean; 585 KiB data, 93 MiB used, 715 GiB / 715 GiB avail; 195 KiB/s rd, 204 B/s wr, 296 op/s 2024-06-01T01:58:16.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:16 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:16.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:16 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:17.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:58:17 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:58:17] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:58:18.294 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:58:17 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:58:17] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:58:18.608 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:58:18 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ::ffff:172.21.15.161 - - [01/Jun/2024:01:58:18] "GET /metrics HTTP/1.1" 200 213397 "" "Prometheus/2.33.4" 2024-06-01T01:58:18.608 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:18 smithi116 ceph-mon[29958]: pgmap v98: 161 pgs: 161 active+clean; 585 KiB data, 93 MiB used, 715 GiB / 715 GiB avail; 195 KiB/s rd, 204 B/s wr, 296 op/s 2024-06-01T01:58:18.608 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:18 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:18.608 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:18 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:18.609 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:18 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:18.609 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:18 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:18.609 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:18 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:58:18.609 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:18 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:58:18.609 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:18 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:18.609 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:18 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi116.gjuxzb", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-01T01:58:18.609 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:18 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi116.gjuxzb", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-06-01T01:58:18.609 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:18 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:58:18.609 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:18 smithi116 ceph-mon[25082]: pgmap v98: 161 pgs: 161 active+clean; 585 KiB data, 93 MiB used, 715 GiB / 715 GiB avail; 195 KiB/s rd, 204 B/s wr, 296 op/s 2024-06-01T01:58:18.609 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:18 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:18.609 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:18 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:18.609 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:18 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:18.610 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:18 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:18.610 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:18 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:58:18.610 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:18 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:58:18.610 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:18 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:18.610 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:18 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi116.gjuxzb", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-01T01:58:18.610 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:18 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi116.gjuxzb", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-06-01T01:58:18.610 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:18 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:58:18.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:18 smithi161 ceph-mon[26055]: pgmap v98: 161 pgs: 161 active+clean; 585 KiB data, 93 MiB used, 715 GiB / 715 GiB avail; 195 KiB/s rd, 204 B/s wr, 296 op/s 2024-06-01T01:58:18.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:18 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:18.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:18 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:18.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:18 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:18.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:18 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:18.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:18 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:58:18.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:18 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:58:18.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:18 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:18.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:18 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi116.gjuxzb", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-01T01:58:18.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:18 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi116.gjuxzb", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-06-01T01:58:18.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:18 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:58:19.644 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:19 smithi116 ceph-mon[25082]: Checking pool "foo" exists for service iscsi.foo 2024-06-01T01:58:19.644 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:19 smithi116 ceph-mon[25082]: Deploying daemon iscsi.foo.smithi116.gjuxzb on smithi116 2024-06-01T01:58:19.645 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:19 smithi116 ceph-mon[29958]: Checking pool "foo" exists for service iscsi.foo 2024-06-01T01:58:19.645 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:19 smithi116 ceph-mon[29958]: Deploying daemon iscsi.foo.smithi116.gjuxzb on smithi116 2024-06-01T01:58:19.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:19 smithi161 ceph-mon[26055]: Checking pool "foo" exists for service iscsi.foo 2024-06-01T01:58:19.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:19 smithi161 ceph-mon[26055]: Deploying daemon iscsi.foo.smithi116.gjuxzb on smithi116 2024-06-01T01:58:20.563 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:20 smithi116 ceph-mon[29958]: pgmap v99: 161 pgs: 161 active+clean; 585 KiB data, 93 MiB used, 715 GiB / 715 GiB avail; 216 KiB/s rd, 204 B/s wr, 327 op/s 2024-06-01T01:58:20.563 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:20 smithi116 ceph-mon[29958]: mgrmap e20: y(active, since 2m), standbys: x 2024-06-01T01:58:20.563 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:20 smithi116 ceph-mon[25082]: pgmap v99: 161 pgs: 161 active+clean; 585 KiB data, 93 MiB used, 715 GiB / 715 GiB avail; 216 KiB/s rd, 204 B/s wr, 327 op/s 2024-06-01T01:58:20.564 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:20 smithi116 ceph-mon[25082]: mgrmap e20: y(active, since 2m), standbys: x 2024-06-01T01:58:20.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:20 smithi161 ceph-mon[26055]: pgmap v99: 161 pgs: 161 active+clean; 585 KiB data, 93 MiB used, 715 GiB / 715 GiB avail; 216 KiB/s rd, 204 B/s wr, 327 op/s 2024-06-01T01:58:20.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:20 smithi161 ceph-mon[26055]: mgrmap e20: y(active, since 2m), standbys: x 2024-06-01T01:58:22.199 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[29958]: pgmap v100: 161 pgs: 161 active+clean; 585 KiB data, 93 MiB used, 715 GiB / 715 GiB avail; 185 KiB/s rd, 178 B/s wr, 280 op/s 2024-06-01T01:58:22.199 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:22.199 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:58:22.199 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:58:22.199 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:22.199 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:58:22.200 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-06-01T01:58:22.200 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[29958]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-06-01T01:58:22.200 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[29958]: Adding iSCSI gateway http://:@172.21.15.116:5000 to Dashboard 2024-06-01T01:58:22.200 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-06-01T01:58:22.200 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[29958]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-06-01T01:58:22.200 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:22.200 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi116"}]: dispatch 2024-06-01T01:58:22.200 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[29958]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi116"}]: dispatch 2024-06-01T01:58:22.200 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:22.200 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:58:22.200 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:58:22.200 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:22.200 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:58:22.200 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/3363830819' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-01T01:58:22.200 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1763108185' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3390453904"}]: dispatch 2024-06-01T01:58:22.201 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[29958]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3390453904"}]: dispatch 2024-06-01T01:58:22.201 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[25082]: pgmap v100: 161 pgs: 161 active+clean; 585 KiB data, 93 MiB used, 715 GiB / 715 GiB avail; 185 KiB/s rd, 178 B/s wr, 280 op/s 2024-06-01T01:58:22.201 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:22.201 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:58:22.201 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:58:22.201 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:22.201 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:58:22.201 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-06-01T01:58:22.201 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[25082]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-06-01T01:58:22.201 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[25082]: Adding iSCSI gateway http://:@172.21.15.116:5000 to Dashboard 2024-06-01T01:58:22.201 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-06-01T01:58:22.201 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[25082]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-06-01T01:58:22.201 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:22.202 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi116"}]: dispatch 2024-06-01T01:58:22.202 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[25082]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi116"}]: dispatch 2024-06-01T01:58:22.202 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:22.202 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:58:22.202 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:58:22.202 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:22.202 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:58:22.202 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/3363830819' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-01T01:58:22.202 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1763108185' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3390453904"}]: dispatch 2024-06-01T01:58:22.202 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:22 smithi116 ceph-mon[25082]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3390453904"}]: dispatch 2024-06-01T01:58:22.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:22 smithi161 ceph-mon[26055]: pgmap v100: 161 pgs: 161 active+clean; 585 KiB data, 93 MiB used, 715 GiB / 715 GiB avail; 185 KiB/s rd, 178 B/s wr, 280 op/s 2024-06-01T01:58:22.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:22 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:22.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:22 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:58:22.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:22 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:58:22.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:22 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:22.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:22 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:58:22.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:22 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-06-01T01:58:22.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:22 smithi161 ceph-mon[26055]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-06-01T01:58:22.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:22 smithi161 ceph-mon[26055]: Adding iSCSI gateway http://:@172.21.15.116:5000 to Dashboard 2024-06-01T01:58:22.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:22 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-06-01T01:58:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:22 smithi161 ceph-mon[26055]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-06-01T01:58:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:22 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:22 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi116"}]: dispatch 2024-06-01T01:58:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:22 smithi161 ceph-mon[26055]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi116"}]: dispatch 2024-06-01T01:58:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:22 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:22 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:58:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:22 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:58:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:22 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:22 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T01:58:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:22 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/3363830819' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-01T01:58:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:22 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1763108185' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3390453904"}]: dispatch 2024-06-01T01:58:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:22 smithi161 ceph-mon[26055]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3390453904"}]: dispatch 2024-06-01T01:58:23.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:23 smithi161 ceph-mon[26055]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3390453904"}]': finished 2024-06-01T01:58:23.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:23 smithi161 ceph-mon[26055]: osdmap e78: 8 total, 8 up, 8 in 2024-06-01T01:58:23.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:23 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/2371298198' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3769678136"}]: dispatch 2024-06-01T01:58:23.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:23 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:23.704 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:23 smithi116 ceph-mon[25082]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3390453904"}]': finished 2024-06-01T01:58:23.704 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:23 smithi116 ceph-mon[25082]: osdmap e78: 8 total, 8 up, 8 in 2024-06-01T01:58:23.704 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:23 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/2371298198' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3769678136"}]: dispatch 2024-06-01T01:58:23.704 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:23 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:23.705 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:23 smithi116 ceph-mon[29958]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3390453904"}]': finished 2024-06-01T01:58:23.705 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:23 smithi116 ceph-mon[29958]: osdmap e78: 8 total, 8 up, 8 in 2024-06-01T01:58:23.705 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:23 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/2371298198' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3769678136"}]: dispatch 2024-06-01T01:58:23.705 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:23 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:24.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:24 smithi116 ceph-mon[29958]: pgmap v102: 161 pgs: 161 active+clean; 585 KiB data, 93 MiB used, 715 GiB / 715 GiB avail; 213 KiB/s rd, 204 B/s wr, 321 op/s 2024-06-01T01:58:24.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:24 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/2371298198' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3769678136"}]': finished 2024-06-01T01:58:24.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:24 smithi116 ceph-mon[29958]: osdmap e79: 8 total, 8 up, 8 in 2024-06-01T01:58:24.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:24 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1415991254' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/3557119511"}]: dispatch 2024-06-01T01:58:24.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:24 smithi116 ceph-mon[29958]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/3557119511"}]: dispatch 2024-06-01T01:58:24.668 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:24 smithi116 ceph-mon[25082]: pgmap v102: 161 pgs: 161 active+clean; 585 KiB data, 93 MiB used, 715 GiB / 715 GiB avail; 213 KiB/s rd, 204 B/s wr, 321 op/s 2024-06-01T01:58:24.668 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:24 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/2371298198' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3769678136"}]': finished 2024-06-01T01:58:24.668 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:24 smithi116 ceph-mon[25082]: osdmap e79: 8 total, 8 up, 8 in 2024-06-01T01:58:24.668 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:24 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1415991254' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/3557119511"}]: dispatch 2024-06-01T01:58:24.668 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:24 smithi116 ceph-mon[25082]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/3557119511"}]: dispatch 2024-06-01T01:58:24.695 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:58:24 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: ::ffff:172.21.15.161 - - [01/Jun/2024:01:58:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-06-01T01:58:24.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:24 smithi161 ceph-mon[26055]: pgmap v102: 161 pgs: 161 active+clean; 585 KiB data, 93 MiB used, 715 GiB / 715 GiB avail; 213 KiB/s rd, 204 B/s wr, 321 op/s 2024-06-01T01:58:24.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:24 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/2371298198' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3769678136"}]': finished 2024-06-01T01:58:24.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:24 smithi161 ceph-mon[26055]: osdmap e79: 8 total, 8 up, 8 in 2024-06-01T01:58:24.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:24 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1415991254' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/3557119511"}]: dispatch 2024-06-01T01:58:24.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:24 smithi161 ceph-mon[26055]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/3557119511"}]: dispatch 2024-06-01T01:58:25.044 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:58:24 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:58:24] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:58:25.659 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:25 smithi116 ceph-mon[29958]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/3557119511"}]': finished 2024-06-01T01:58:25.659 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:25 smithi116 ceph-mon[29958]: osdmap e80: 8 total, 8 up, 8 in 2024-06-01T01:58:25.659 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:25 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/3157977737' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/1936404193"}]: dispatch 2024-06-01T01:58:25.659 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:25 smithi116 ceph-mon[29958]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/1936404193"}]: dispatch 2024-06-01T01:58:25.659 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:25 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:25.660 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:25 smithi116 ceph-mon[25082]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/3557119511"}]': finished 2024-06-01T01:58:25.660 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:25 smithi116 ceph-mon[25082]: osdmap e80: 8 total, 8 up, 8 in 2024-06-01T01:58:25.660 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:25 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/3157977737' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/1936404193"}]: dispatch 2024-06-01T01:58:25.660 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:25 smithi116 ceph-mon[25082]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/1936404193"}]: dispatch 2024-06-01T01:58:25.660 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:25 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:25.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:25 smithi161 ceph-mon[26055]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/3557119511"}]': finished 2024-06-01T01:58:25.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:25 smithi161 ceph-mon[26055]: osdmap e80: 8 total, 8 up, 8 in 2024-06-01T01:58:25.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:25 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/3157977737' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/1936404193"}]: dispatch 2024-06-01T01:58:25.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:25 smithi161 ceph-mon[26055]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/1936404193"}]: dispatch 2024-06-01T01:58:25.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:25 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:26.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:26 smithi116 ceph-mon[29958]: pgmap v105: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 170 B/s wr, 3 op/s 2024-06-01T01:58:26.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:26 smithi116 ceph-mon[29958]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/1936404193"}]': finished 2024-06-01T01:58:26.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:26 smithi116 ceph-mon[29958]: osdmap e81: 8 total, 8 up, 8 in 2024-06-01T01:58:26.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:26 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/696197458' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/3557119511"}]: dispatch 2024-06-01T01:58:26.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:26 smithi116 ceph-mon[25082]: pgmap v105: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 170 B/s wr, 3 op/s 2024-06-01T01:58:26.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:26 smithi116 ceph-mon[25082]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/1936404193"}]': finished 2024-06-01T01:58:26.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:26 smithi116 ceph-mon[25082]: osdmap e81: 8 total, 8 up, 8 in 2024-06-01T01:58:26.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:26 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/696197458' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/3557119511"}]: dispatch 2024-06-01T01:58:26.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:26 smithi161 ceph-mon[26055]: pgmap v105: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 170 B/s wr, 3 op/s 2024-06-01T01:58:26.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:26 smithi161 ceph-mon[26055]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/1936404193"}]': finished 2024-06-01T01:58:26.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:26 smithi161 ceph-mon[26055]: osdmap e81: 8 total, 8 up, 8 in 2024-06-01T01:58:26.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:26 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/696197458' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/3557119511"}]: dispatch 2024-06-01T01:58:27.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:27 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/696197458' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/3557119511"}]': finished 2024-06-01T01:58:27.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:27 smithi161 ceph-mon[26055]: osdmap e82: 8 total, 8 up, 8 in 2024-06-01T01:58:27.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:27 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/4080059360' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/2080484873"}]: dispatch 2024-06-01T01:58:27.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:27 smithi161 ceph-mon[26055]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/2080484873"}]: dispatch 2024-06-01T01:58:27.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:58:27 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:58:27] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:58:27.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:27 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/696197458' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/3557119511"}]': finished 2024-06-01T01:58:27.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:27 smithi116 ceph-mon[25082]: osdmap e82: 8 total, 8 up, 8 in 2024-06-01T01:58:27.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:27 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/4080059360' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/2080484873"}]: dispatch 2024-06-01T01:58:27.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:27 smithi116 ceph-mon[25082]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/2080484873"}]: dispatch 2024-06-01T01:58:27.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:27 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/696197458' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/3557119511"}]': finished 2024-06-01T01:58:27.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:27 smithi116 ceph-mon[29958]: osdmap e82: 8 total, 8 up, 8 in 2024-06-01T01:58:27.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:27 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/4080059360' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/2080484873"}]: dispatch 2024-06-01T01:58:27.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:27 smithi116 ceph-mon[29958]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/2080484873"}]: dispatch 2024-06-01T01:58:28.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:28 smithi161 ceph-mon[26055]: pgmap v108: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-01T01:58:28.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:28 smithi161 ceph-mon[26055]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/2080484873"}]': finished 2024-06-01T01:58:28.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:28 smithi161 ceph-mon[26055]: osdmap e83: 8 total, 8 up, 8 in 2024-06-01T01:58:28.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:28 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:28.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:28 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:28.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:28 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:58:28.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:28 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:58:28.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:28 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:28.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:28 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/3946190564' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3719895177"}]: dispatch 2024-06-01T01:58:28.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:58:28 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ::ffff:172.21.15.161 - - [01/Jun/2024:01:58:28] "GET /metrics HTTP/1.1" 200 215016 "" "Prometheus/2.33.4" 2024-06-01T01:58:28.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:28 smithi116 ceph-mon[25082]: pgmap v108: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-01T01:58:28.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:28 smithi116 ceph-mon[25082]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/2080484873"}]': finished 2024-06-01T01:58:28.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:28 smithi116 ceph-mon[25082]: osdmap e83: 8 total, 8 up, 8 in 2024-06-01T01:58:28.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:28 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:28.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:28 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:28.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:28 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:58:28.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:28 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:58:28.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:28 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:28.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:28 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/3946190564' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3719895177"}]: dispatch 2024-06-01T01:58:28.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:28 smithi116 ceph-mon[29958]: pgmap v108: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-01T01:58:28.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:28 smithi116 ceph-mon[29958]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/2080484873"}]': finished 2024-06-01T01:58:28.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:28 smithi116 ceph-mon[29958]: osdmap e83: 8 total, 8 up, 8 in 2024-06-01T01:58:28.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:28 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:28.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:28 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:28.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:28 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:58:28.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:28 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:58:28.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:28 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:28.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:28 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/3946190564' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3719895177"}]: dispatch 2024-06-01T01:58:29.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:29 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/3946190564' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3719895177"}]': finished 2024-06-01T01:58:29.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:29 smithi116 ceph-mon[29958]: osdmap e84: 8 total, 8 up, 8 in 2024-06-01T01:58:29.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:29 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1882662388' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/4034720111"}]: dispatch 2024-06-01T01:58:29.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:29 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/3946190564' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3719895177"}]': finished 2024-06-01T01:58:29.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:29 smithi116 ceph-mon[25082]: osdmap e84: 8 total, 8 up, 8 in 2024-06-01T01:58:29.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:29 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1882662388' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/4034720111"}]: dispatch 2024-06-01T01:58:29.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:29 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/3946190564' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3719895177"}]': finished 2024-06-01T01:58:29.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:29 smithi161 ceph-mon[26055]: osdmap e84: 8 total, 8 up, 8 in 2024-06-01T01:58:29.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:29 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1882662388' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/4034720111"}]: dispatch 2024-06-01T01:58:30.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:30 smithi116 ceph-mon[25082]: pgmap v111: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:58:30.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:30 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1882662388' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/4034720111"}]': finished 2024-06-01T01:58:30.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:30 smithi116 ceph-mon[25082]: osdmap e85: 8 total, 8 up, 8 in 2024-06-01T01:58:30.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:30 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/2914234540' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1985344094"}]: dispatch 2024-06-01T01:58:30.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:30 smithi116 ceph-mon[25082]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1985344094"}]: dispatch 2024-06-01T01:58:30.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:30 smithi116 ceph-mon[29958]: pgmap v111: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:58:30.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:30 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1882662388' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/4034720111"}]': finished 2024-06-01T01:58:30.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:30 smithi116 ceph-mon[29958]: osdmap e85: 8 total, 8 up, 8 in 2024-06-01T01:58:30.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:30 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/2914234540' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1985344094"}]: dispatch 2024-06-01T01:58:30.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:30 smithi116 ceph-mon[29958]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1985344094"}]: dispatch 2024-06-01T01:58:30.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:30 smithi161 ceph-mon[26055]: pgmap v111: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:58:30.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:30 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1882662388' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/4034720111"}]': finished 2024-06-01T01:58:30.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:30 smithi161 ceph-mon[26055]: osdmap e85: 8 total, 8 up, 8 in 2024-06-01T01:58:30.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:30 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/2914234540' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1985344094"}]: dispatch 2024-06-01T01:58:30.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:30 smithi161 ceph-mon[26055]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1985344094"}]: dispatch 2024-06-01T01:58:31.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:31 smithi116 ceph-mon[25082]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1985344094"}]': finished 2024-06-01T01:58:31.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:31 smithi116 ceph-mon[25082]: osdmap e86: 8 total, 8 up, 8 in 2024-06-01T01:58:31.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:31 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/3745610463' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/455415098"}]: dispatch 2024-06-01T01:58:31.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:31 smithi116 ceph-mon[25082]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/455415098"}]: dispatch 2024-06-01T01:58:31.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:31 smithi116 ceph-mon[29958]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1985344094"}]': finished 2024-06-01T01:58:31.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:31 smithi116 ceph-mon[29958]: osdmap e86: 8 total, 8 up, 8 in 2024-06-01T01:58:31.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:31 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/3745610463' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/455415098"}]: dispatch 2024-06-01T01:58:31.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:31 smithi116 ceph-mon[29958]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/455415098"}]: dispatch 2024-06-01T01:58:31.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:31 smithi161 ceph-mon[26055]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1985344094"}]': finished 2024-06-01T01:58:31.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:31 smithi161 ceph-mon[26055]: osdmap e86: 8 total, 8 up, 8 in 2024-06-01T01:58:31.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:31 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/3745610463' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/455415098"}]: dispatch 2024-06-01T01:58:31.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:31 smithi161 ceph-mon[26055]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/455415098"}]: dispatch 2024-06-01T01:58:32.750 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:32 smithi161 ceph-mon[26055]: pgmap v114: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:58:32.750 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:32 smithi161 ceph-mon[26055]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T01:58:32.750 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:32 smithi161 ceph-mon[26055]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/455415098"}]': finished 2024-06-01T01:58:32.751 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:32 smithi161 ceph-mon[26055]: osdmap e87: 8 total, 8 up, 8 in 2024-06-01T01:58:32.751 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:32 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/3003371833' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3329906888"}]: dispatch 2024-06-01T01:58:32.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:32 smithi116 ceph-mon[25082]: pgmap v114: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:58:32.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:32 smithi116 ceph-mon[25082]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T01:58:32.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:32 smithi116 ceph-mon[25082]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/455415098"}]': finished 2024-06-01T01:58:32.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:32 smithi116 ceph-mon[25082]: osdmap e87: 8 total, 8 up, 8 in 2024-06-01T01:58:32.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:32 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/3003371833' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3329906888"}]: dispatch 2024-06-01T01:58:32.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:32 smithi116 ceph-mon[29958]: pgmap v114: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:58:32.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:32 smithi116 ceph-mon[29958]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T01:58:32.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:32 smithi116 ceph-mon[29958]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/455415098"}]': finished 2024-06-01T01:58:32.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:32 smithi116 ceph-mon[29958]: osdmap e87: 8 total, 8 up, 8 in 2024-06-01T01:58:32.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:32 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/3003371833' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3329906888"}]: dispatch 2024-06-01T01:58:33.753 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:33 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/3003371833' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3329906888"}]': finished 2024-06-01T01:58:33.754 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:33 smithi161 ceph-mon[26055]: osdmap e88: 8 total, 8 up, 8 in 2024-06-01T01:58:33.754 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:33 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/4138647810' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3120590326"}]: dispatch 2024-06-01T01:58:33.754 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:33 smithi161 ceph-mon[26055]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3120590326"}]: dispatch 2024-06-01T01:58:33.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:33 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/3003371833' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3329906888"}]': finished 2024-06-01T01:58:33.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:33 smithi116 ceph-mon[25082]: osdmap e88: 8 total, 8 up, 8 in 2024-06-01T01:58:33.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:33 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/4138647810' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3120590326"}]: dispatch 2024-06-01T01:58:33.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:33 smithi116 ceph-mon[25082]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3120590326"}]: dispatch 2024-06-01T01:58:33.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:33 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/3003371833' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3329906888"}]': finished 2024-06-01T01:58:33.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:33 smithi116 ceph-mon[29958]: osdmap e88: 8 total, 8 up, 8 in 2024-06-01T01:58:33.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:33 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/4138647810' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3120590326"}]: dispatch 2024-06-01T01:58:33.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:33 smithi116 ceph-mon[29958]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3120590326"}]: dispatch 2024-06-01T01:58:33.795 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:58:33 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=warn ts=2024-06-01T01:58:33.520Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.116:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.116 because it doesn't contain any IP SANs" 2024-06-01T01:58:33.795 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:58:33 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=warn ts=2024-06-01T01:58:33.520Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.161:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.161 because it doesn't contain any IP SANs" 2024-06-01T01:58:34.559 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:34 smithi161 ceph-mon[26055]: pgmap v117: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:58:34.559 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:34 smithi161 ceph-mon[26055]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3120590326"}]': finished 2024-06-01T01:58:34.559 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:34 smithi161 ceph-mon[26055]: osdmap e89: 8 total, 8 up, 8 in 2024-06-01T01:58:34.559 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:34 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/948077579' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/1936404193"}]: dispatch 2024-06-01T01:58:34.559 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:58:34 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: ::ffff:172.21.15.161 - - [01/Jun/2024:01:58:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-06-01T01:58:34.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:58:34 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:58:34] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:58:34.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:34 smithi116 ceph-mon[25082]: pgmap v117: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:58:34.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:34 smithi116 ceph-mon[25082]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3120590326"}]': finished 2024-06-01T01:58:34.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:34 smithi116 ceph-mon[25082]: osdmap e89: 8 total, 8 up, 8 in 2024-06-01T01:58:34.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:34 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/948077579' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/1936404193"}]: dispatch 2024-06-01T01:58:34.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:34 smithi116 ceph-mon[29958]: pgmap v117: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail 2024-06-01T01:58:34.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:34 smithi116 ceph-mon[29958]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3120590326"}]': finished 2024-06-01T01:58:34.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:34 smithi116 ceph-mon[29958]: osdmap e89: 8 total, 8 up, 8 in 2024-06-01T01:58:34.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:34 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/948077579' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/1936404193"}]: dispatch 2024-06-01T01:58:35.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:35 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/948077579' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/1936404193"}]': finished 2024-06-01T01:58:35.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:35 smithi116 ceph-mon[25082]: osdmap e90: 8 total, 8 up, 8 in 2024-06-01T01:58:35.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:35 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:35.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:35 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:35.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:35 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1296880290' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/2550839719"}]: dispatch 2024-06-01T01:58:35.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:35 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/948077579' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/1936404193"}]': finished 2024-06-01T01:58:35.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:35 smithi116 ceph-mon[29958]: osdmap e90: 8 total, 8 up, 8 in 2024-06-01T01:58:35.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:35 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:35.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:35 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:35.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:35 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1296880290' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/2550839719"}]: dispatch 2024-06-01T01:58:35.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:35 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/948077579' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/1936404193"}]': finished 2024-06-01T01:58:35.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:35 smithi161 ceph-mon[26055]: osdmap e90: 8 total, 8 up, 8 in 2024-06-01T01:58:35.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:35 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:35.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:35 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:35.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:35 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1296880290' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/2550839719"}]: dispatch 2024-06-01T01:58:36.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:36 smithi161 ceph-mon[26055]: pgmap v120: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:58:36.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:36 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1296880290' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/2550839719"}]': finished 2024-06-01T01:58:36.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:36 smithi161 ceph-mon[26055]: osdmap e91: 8 total, 8 up, 8 in 2024-06-01T01:58:36.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:36 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1811800861' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/2138133985"}]: dispatch 2024-06-01T01:58:36.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:36 smithi161 ceph-mon[26055]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/2138133985"}]: dispatch 2024-06-01T01:58:37.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:36 smithi116 ceph-mon[25082]: pgmap v120: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:58:37.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:36 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1296880290' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/2550839719"}]': finished 2024-06-01T01:58:37.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:36 smithi116 ceph-mon[25082]: osdmap e91: 8 total, 8 up, 8 in 2024-06-01T01:58:37.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:36 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1811800861' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/2138133985"}]: dispatch 2024-06-01T01:58:37.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:36 smithi116 ceph-mon[25082]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/2138133985"}]: dispatch 2024-06-01T01:58:37.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:36 smithi116 ceph-mon[29958]: pgmap v120: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:58:37.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:36 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1296880290' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/2550839719"}]': finished 2024-06-01T01:58:37.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:36 smithi116 ceph-mon[29958]: osdmap e91: 8 total, 8 up, 8 in 2024-06-01T01:58:37.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:36 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1811800861' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/2138133985"}]: dispatch 2024-06-01T01:58:37.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:36 smithi116 ceph-mon[29958]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/2138133985"}]: dispatch 2024-06-01T01:58:37.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:37 smithi161 ceph-mon[26055]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/2138133985"}]': finished 2024-06-01T01:58:37.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:37 smithi161 ceph-mon[26055]: osdmap e92: 8 total, 8 up, 8 in 2024-06-01T01:58:37.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:37 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1859615737' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/2138133985"}]: dispatch 2024-06-01T01:58:38.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:37 smithi116 ceph-mon[25082]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/2138133985"}]': finished 2024-06-01T01:58:38.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:37 smithi116 ceph-mon[25082]: osdmap e92: 8 total, 8 up, 8 in 2024-06-01T01:58:38.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:37 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1859615737' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/2138133985"}]: dispatch 2024-06-01T01:58:38.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:37 smithi116 ceph-mon[29958]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/2138133985"}]': finished 2024-06-01T01:58:38.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:37 smithi116 ceph-mon[29958]: osdmap e92: 8 total, 8 up, 8 in 2024-06-01T01:58:38.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:37 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1859615737' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/2138133985"}]: dispatch 2024-06-01T01:58:38.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:58:38 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ::ffff:172.21.15.161 - - [01/Jun/2024:01:58:38] "GET /metrics HTTP/1.1" 200 215016 "" "Prometheus/2.33.4" 2024-06-01T01:58:38.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:38 smithi116 ceph-mon[25082]: pgmap v123: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:58:38.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:38 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1859615737' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/2138133985"}]': finished 2024-06-01T01:58:38.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:38 smithi116 ceph-mon[25082]: osdmap e93: 8 total, 8 up, 8 in 2024-06-01T01:58:38.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:38 smithi116 ceph-mon[29958]: pgmap v123: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:58:38.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:38 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1859615737' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/2138133985"}]': finished 2024-06-01T01:58:38.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:38 smithi116 ceph-mon[29958]: osdmap e93: 8 total, 8 up, 8 in 2024-06-01T01:58:38.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:38 smithi161 ceph-mon[26055]: pgmap v123: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:58:38.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:38 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1859615737' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/2138133985"}]': finished 2024-06-01T01:58:38.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:38 smithi161 ceph-mon[26055]: osdmap e93: 8 total, 8 up, 8 in 2024-06-01T01:58:40.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:40 smithi161 ceph-mon[26055]: pgmap v125: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:58:41.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:40 smithi116 ceph-mon[25082]: pgmap v125: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:58:41.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:40 smithi116 ceph-mon[29958]: pgmap v125: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:58:41.915 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:58:41 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:58:41] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:58:42.742 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:42 smithi116 ceph-mon[25082]: pgmap v126: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:58:42.743 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:42 smithi116 ceph-mon[25082]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T01:58:42.743 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:42 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:42.743 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:42 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:42.743 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:42 smithi116 ceph-mon[29958]: pgmap v126: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:58:42.743 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:42 smithi116 ceph-mon[29958]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T01:58:42.743 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:42 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:42.743 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:42 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:42.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:42 smithi161 ceph-mon[26055]: pgmap v126: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:58:42.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:42 smithi161 ceph-mon[26055]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T01:58:42.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:42 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:42.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:42 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:43.776 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2024-06-01T01:58:43.776 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-06-01T01:58:43.776 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-06-01T01:58:43.777 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2024-06-01T01:58:43.777 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T01:58:43.777 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T01:58:43.777 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2024-06-01T01:58:43.777 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-06-01T01:58:43.777 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-06-01T01:58:43.777 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2024-06-01T01:58:43.777 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T01:58:43.777 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T01:58:43.778 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 01:58:43 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=error ts=2024-06-01T01:58:43.506Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=6 err="ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://172.21.15.116:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.116 because it doesn't contain any IP SANs; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://172.21.15.161:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.161 because it doesn't contain any IP SANs" 2024-06-01T01:58:43.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2024-06-01T01:58:43.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-06-01T01:58:43.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-06-01T01:58:43.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2024-06-01T01:58:43.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T01:58:43.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T01:58:44.656 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:58:44 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: ::ffff:172.21.15.161 - - [01/Jun/2024:01:58:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-06-01T01:58:44.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:44 smithi116 ceph-mon[25082]: pgmap v127: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 709 B/s rd, 0 op/s 2024-06-01T01:58:44.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:44 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]': finished 2024-06-01T01:58:44.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:44 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]': finished 2024-06-01T01:58:44.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:44 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-06-01T01:58:44.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:44 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]': finished 2024-06-01T01:58:44.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:44 smithi116 ceph-mon[25082]: osdmap e94: 8 total, 8 up, 8 in 2024-06-01T01:58:44.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:44 smithi116 ceph-mon[29958]: pgmap v127: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 709 B/s rd, 0 op/s 2024-06-01T01:58:44.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:44 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]': finished 2024-06-01T01:58:44.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:44 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]': finished 2024-06-01T01:58:44.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:44 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-06-01T01:58:44.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:44 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]': finished 2024-06-01T01:58:44.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:44 smithi116 ceph-mon[29958]: osdmap e94: 8 total, 8 up, 8 in 2024-06-01T01:58:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:44 smithi161 ceph-mon[26055]: pgmap v127: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 709 B/s rd, 0 op/s 2024-06-01T01:58:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:44 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]': finished 2024-06-01T01:58:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:44 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]': finished 2024-06-01T01:58:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:44 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-06-01T01:58:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:44 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]': finished 2024-06-01T01:58:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:44 smithi161 ceph-mon[26055]: osdmap e94: 8 total, 8 up, 8 in 2024-06-01T01:58:45.939 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:45 smithi116 ceph-mon[29958]: osdmap e95: 8 total, 8 up, 8 in 2024-06-01T01:58:45.940 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:45 smithi116 ceph-mon[25082]: osdmap e95: 8 total, 8 up, 8 in 2024-06-01T01:58:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:45 smithi161 ceph-mon[26055]: osdmap e95: 8 total, 8 up, 8 in 2024-06-01T01:58:46.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:46 smithi161 ceph-mon[26055]: pgmap v130: 161 pgs: 4 peering, 157 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-06-01T01:58:47.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:46 smithi116 ceph-mon[25082]: pgmap v130: 161 pgs: 4 peering, 157 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-06-01T01:58:47.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:46 smithi116 ceph-mon[29958]: pgmap v130: 161 pgs: 4 peering, 157 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-06-01T01:58:48.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:48 smithi116 ceph-mon[25082]: pgmap v131: 161 pgs: 4 peering, 157 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-01T01:58:48.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:48 smithi116 ceph-mon[29958]: pgmap v131: 161 pgs: 4 peering, 157 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-01T01:58:48.795 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:58:48 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ::ffff:172.21.15.161 - - [01/Jun/2024:01:58:48] "GET /metrics HTTP/1.1" 200 215033 "" "Prometheus/2.33.4" 2024-06-01T01:58:48.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:48 smithi161 ceph-mon[26055]: pgmap v131: 161 pgs: 4 peering, 157 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-01T01:58:50.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:50 smithi161 ceph-mon[26055]: pgmap v132: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-06-01T01:58:51.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:50 smithi116 ceph-mon[25082]: pgmap v132: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-06-01T01:58:51.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:50 smithi116 ceph-mon[29958]: pgmap v132: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-06-01T01:58:52.866 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:52 smithi161 ceph-mon[26055]: pgmap v133: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-06-01T01:58:52.866 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:52 smithi161 ceph-mon[26055]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T01:58:53.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:52 smithi116 ceph-mon[25082]: pgmap v133: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-06-01T01:58:53.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:52 smithi116 ceph-mon[25082]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T01:58:53.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:52 smithi116 ceph-mon[29958]: pgmap v133: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-06-01T01:58:53.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:52 smithi116 ceph-mon[29958]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T01:58:54.648 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:58:54 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: ::ffff:172.21.15.161 - - [01/Jun/2024:01:58:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-06-01T01:58:54.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:54 smithi161 ceph-mon[26055]: pgmap v134: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-06-01T01:58:54.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:54 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:54.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:54 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:55.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:54 smithi116 ceph-mon[25082]: pgmap v134: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-06-01T01:58:55.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:54 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:55.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:54 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:55.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:54 smithi116 ceph-mon[29958]: pgmap v134: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-06-01T01:58:55.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:54 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:55.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:54 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:58:55.045 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:58:54 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:58:54] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:58:56.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:56 smithi161 ceph-mon[26055]: pgmap v135: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1010 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-01T01:58:57.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:56 smithi116 ceph-mon[25082]: pgmap v135: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1010 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-01T01:58:57.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:56 smithi116 ceph-mon[29958]: pgmap v135: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1010 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-01T01:58:58.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:58:58 smithi116 ceph-mon[29958]: pgmap v136: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-01T01:58:58.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:58:58 smithi116 ceph-mon[25082]: pgmap v136: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-01T01:58:58.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:58:58 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ::ffff:172.21.15.161 - - [01/Jun/2024:01:58:58] "GET /metrics HTTP/1.1" 200 214993 "" "Prometheus/2.33.4" 2024-06-01T01:58:58.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:58:58 smithi161 ceph-mon[26055]: pgmap v136: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-01T01:59:00.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:00 smithi161 ceph-mon[26055]: pgmap v137: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-06-01T01:59:01.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:00 smithi116 ceph-mon[25082]: pgmap v137: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-06-01T01:59:01.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:00 smithi116 ceph-mon[29958]: pgmap v137: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-06-01T01:59:01.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:59:01 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:59:01] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:59:02.741 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:02 smithi116 ceph-mon[25082]: pgmap v138: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:03.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:02 smithi116 ceph-mon[25082]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T01:59:03.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:02 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:03.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:02 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:03.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:02 smithi116 ceph-mon[29958]: pgmap v138: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:03.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:02 smithi116 ceph-mon[29958]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T01:59:03.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:02 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:03.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:02 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:03.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:02 smithi161 ceph-mon[26055]: pgmap v138: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:03.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:02 smithi161 ceph-mon[26055]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T01:59:03.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:02 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:03.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:02 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:04.695 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:59:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: ::ffff:172.21.15.161 - - [01/Jun/2024:01:59:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-06-01T01:59:05.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:04 smithi116 ceph-mon[29958]: pgmap v139: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:05.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:04 smithi116 ceph-mon[25082]: pgmap v139: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:05.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:04 smithi161 ceph-mon[26055]: pgmap v139: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:07.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:06 smithi116 ceph-mon[25082]: pgmap v140: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:07.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:06 smithi116 ceph-mon[29958]: pgmap v140: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:07.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:06 smithi161 ceph-mon[26055]: pgmap v140: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:08.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:07 smithi116 ceph-mon[25082]: pgmap v141: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:08.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:07 smithi116 ceph-mon[29958]: pgmap v141: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:08.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:07 smithi161 ceph-mon[26055]: pgmap v141: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:08.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:59:08 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ::ffff:172.21.15.161 - - [01/Jun/2024:01:59:08] "GET /metrics HTTP/1.1" 200 214993 "" "Prometheus/2.33.4" 2024-06-01T01:59:10.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:10 smithi161 ceph-mon[26055]: pgmap v142: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:10.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:10 smithi116 ceph-mon[25082]: pgmap v142: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:10.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:10 smithi116 ceph-mon[29958]: pgmap v142: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:12.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:12 smithi161 ceph-mon[26055]: pgmap v143: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:12.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:12 smithi161 ceph-mon[26055]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T01:59:12.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:12 smithi116 ceph-mon[25082]: pgmap v143: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:12.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:12 smithi116 ceph-mon[25082]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T01:59:12.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:12 smithi116 ceph-mon[29958]: pgmap v143: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:12.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:12 smithi116 ceph-mon[29958]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T01:59:14.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:14 smithi116 ceph-mon[25082]: pgmap v144: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:14.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:14 smithi116 ceph-mon[29958]: pgmap v144: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:14.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:14 smithi161 ceph-mon[26055]: pgmap v144: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:14.695 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:59:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: ::ffff:172.21.15.161 - - [01/Jun/2024:01:59:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-06-01T01:59:15.044 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:59:14 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:59:14] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:59:15.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:15 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:15.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:15 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:15.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:15 smithi161 ceph-mon[26055]: pgmap v145: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:16.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:15 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:16.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:15 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:16.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:15 smithi116 ceph-mon[25082]: pgmap v145: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:16.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:15 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:16.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:15 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:16.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:15 smithi116 ceph-mon[29958]: pgmap v145: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:18.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:17 smithi161 ceph-mon[26055]: pgmap v146: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:18.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:17 smithi116 ceph-mon[25082]: pgmap v146: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:18.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:17 smithi116 ceph-mon[29958]: pgmap v146: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:18.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:59:18 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ::ffff:172.21.15.161 - - [01/Jun/2024:01:59:18] "GET /metrics HTTP/1.1" 200 214970 "" "Prometheus/2.33.4" 2024-06-01T01:59:20.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:20 smithi161 ceph-mon[26055]: pgmap v147: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:20.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:20 smithi116 ceph-mon[25082]: pgmap v147: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:20.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:20 smithi116 ceph-mon[29958]: pgmap v147: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:22.294 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:59:21 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:59:21] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:59:22.598 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:22 smithi116 ceph-mon[25082]: pgmap v148: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:22.599 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:22 smithi116 ceph-mon[25082]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T01:59:22.599 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:22 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:22.599 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:22 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:22.599 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:22 smithi116 ceph-mon[29958]: pgmap v148: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:22.599 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:22 smithi116 ceph-mon[29958]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T01:59:22.599 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:22 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:22.599 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:22 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:22.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:22 smithi161 ceph-mon[26055]: pgmap v148: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:22.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:22 smithi161 ceph-mon[26055]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T01:59:22.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:22 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:22.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:22 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:24.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:24 smithi161 ceph-mon[26055]: pgmap v149: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:24.696 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:59:24 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: ::ffff:172.21.15.161 - - [01/Jun/2024:01:59:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-06-01T01:59:24.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:24 smithi116 ceph-mon[29958]: pgmap v149: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:24.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:24 smithi116 ceph-mon[25082]: pgmap v149: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:26.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:26 smithi161 ceph-mon[26055]: pgmap v150: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:26.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:26 smithi116 ceph-mon[25082]: pgmap v150: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:26.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:26 smithi116 ceph-mon[29958]: pgmap v150: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:28.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:28 smithi161 ceph-mon[26055]: pgmap v151: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:28.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:28 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:59:28.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:28 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:59:28.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:28 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:28.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:59:28 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ::ffff:172.21.15.161 - - [01/Jun/2024:01:59:28] "GET /metrics HTTP/1.1" 200 214946 "" "Prometheus/2.33.4" 2024-06-01T01:59:28.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:28 smithi116 ceph-mon[25082]: pgmap v151: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:28.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:28 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:59:28.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:28 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:59:28.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:28 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:28.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:28 smithi116 ceph-mon[29958]: pgmap v151: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:28.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:28 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T01:59:28.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:28 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T01:59:28.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:28 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:30.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:30 smithi161 ceph-mon[26055]: pgmap v152: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:30.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:30 smithi116 ceph-mon[25082]: pgmap v152: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:30.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:30 smithi116 ceph-mon[29958]: pgmap v152: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:32.582 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:32 smithi161 ceph-mon[26055]: pgmap v153: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:32.582 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:32 smithi161 ceph-mon[26055]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T01:59:32.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:32 smithi116 ceph-mon[25082]: pgmap v153: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:32.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:32 smithi116 ceph-mon[25082]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T01:59:32.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:32 smithi116 ceph-mon[29958]: pgmap v153: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:32.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:32 smithi116 ceph-mon[29958]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T01:59:34.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:34 smithi161 ceph-mon[26055]: pgmap v154: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:34.445 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:59:34 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: ::ffff:172.21.15.161 - - [01/Jun/2024:01:59:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-06-01T01:59:34.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:34 smithi116 ceph-mon[25082]: pgmap v154: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:34.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:34 smithi116 ceph-mon[29958]: pgmap v154: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:35.294 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:59:34 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:59:34] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:59:36.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:35 smithi161 ceph-mon[26055]: pgmap v155: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:36.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:35 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:36.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:35 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:36.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:35 smithi116 ceph-mon[25082]: pgmap v155: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:36.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:35 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:36.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:35 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:36.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:35 smithi116 ceph-mon[29958]: pgmap v155: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:36.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:35 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:36.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:35 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:38.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:37 smithi161 ceph-mon[26055]: pgmap v156: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:38.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:37 smithi116 ceph-mon[25082]: pgmap v156: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:38.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:37 smithi116 ceph-mon[29958]: pgmap v156: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:38.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:59:38 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ::ffff:172.21.15.161 - - [01/Jun/2024:01:59:38] "GET /metrics HTTP/1.1" 200 214946 "" "Prometheus/2.33.4" 2024-06-01T01:59:40.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:40 smithi161 ceph-mon[26055]: pgmap v157: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:40.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:40 smithi116 ceph-mon[25082]: pgmap v157: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:40.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:40 smithi116 ceph-mon[29958]: pgmap v157: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:42.294 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:59:41 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:01:59:41] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:59:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:42 smithi161 ceph-mon[26055]: pgmap v158: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:42 smithi161 ceph-mon[26055]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T01:59:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:42 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:42 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:42.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:42 smithi116 ceph-mon[29958]: pgmap v158: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:42.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:42 smithi116 ceph-mon[29958]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T01:59:42.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:42 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:42.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:42 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:42.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:42 smithi116 ceph-mon[25082]: pgmap v158: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:42.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:42 smithi116 ceph-mon[25082]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T01:59:42.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:42 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:42.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:42 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:43.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T01:59:43.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T01:59:43.521 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T01:59:43.521 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T01:59:43.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T01:59:43.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T01:59:44.533 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:44 smithi116 ceph-mon[25082]: pgmap v159: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:44.533 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:44 smithi116 ceph-mon[29958]: pgmap v159: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:44.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:44 smithi161 ceph-mon[26055]: pgmap v159: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:44.695 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:59:44 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: ::ffff:172.21.15.161 - - [01/Jun/2024:01:59:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-06-01T01:59:46.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:46 smithi161 ceph-mon[26055]: pgmap v160: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:46.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:46 smithi116 ceph-mon[25082]: pgmap v160: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:46.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:46 smithi116 ceph-mon[29958]: pgmap v160: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:48.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:48 smithi161 ceph-mon[26055]: pgmap v161: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:48.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:48 smithi116 ceph-mon[25082]: pgmap v161: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:48.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:48 smithi116 ceph-mon[29958]: pgmap v161: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:48.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:59:48 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ::ffff:172.21.15.161 - - [01/Jun/2024:01:59:48] "GET /metrics HTTP/1.1" 200 214965 "" "Prometheus/2.33.4" 2024-06-01T01:59:50.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:50 smithi161 ceph-mon[26055]: pgmap v162: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:50.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:50 smithi116 ceph-mon[25082]: pgmap v162: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:50.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:50 smithi116 ceph-mon[29958]: pgmap v162: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:52.596 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:52 smithi161 ceph-mon[26055]: pgmap v163: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:52.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:52 smithi161 ceph-mon[26055]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T01:59:52.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:52 smithi116 ceph-mon[25082]: pgmap v163: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:52.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:52 smithi116 ceph-mon[25082]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T01:59:52.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:52 smithi116 ceph-mon[29958]: pgmap v163: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:52.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:52 smithi116 ceph-mon[29958]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T01:59:54.686 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:54 smithi161 ceph-mon[26055]: pgmap v164: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:54.686 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 01:59:54 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: ::ffff:172.21.15.161 - - [01/Jun/2024:01:59:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-06-01T01:59:54.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:54 smithi116 ceph-mon[25082]: pgmap v164: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:54.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:54 smithi116 ceph-mon[29958]: pgmap v164: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:55.294 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:59:54 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:01:59:54] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T01:59:56.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:55 smithi116 ceph-mon[25082]: pgmap v165: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:56.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:55 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:56.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:55 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:56.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:55 smithi116 ceph-mon[29958]: pgmap v165: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:56.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:55 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:56.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:55 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:56.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:55 smithi161 ceph-mon[26055]: pgmap v165: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T01:59:56.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:55 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:56.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:55 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T01:59:58.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 01:59:57 smithi116 ceph-mon[25082]: pgmap v166: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:58.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 01:59:57 smithi116 ceph-mon[29958]: pgmap v166: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:58.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 01:59:57 smithi161 ceph-mon[26055]: pgmap v166: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T01:59:58.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 01:59:58 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ::ffff:172.21.15.161 - - [01/Jun/2024:01:59:58] "GET /metrics HTTP/1.1" 200 214974 "" "Prometheus/2.33.4" 2024-06-01T02:00:00.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:00 smithi161 ceph-mon[26055]: pgmap v167: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:00.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:00 smithi161 ceph-mon[26055]: overall HEALTH_OK 2024-06-01T02:00:00.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:00 smithi116 ceph-mon[25082]: pgmap v167: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:00.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:00 smithi116 ceph-mon[25082]: overall HEALTH_OK 2024-06-01T02:00:00.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:00 smithi116 ceph-mon[29958]: pgmap v167: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:00.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:00 smithi116 ceph-mon[29958]: overall HEALTH_OK 2024-06-01T02:00:02.316 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:00:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:02:00:02] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T02:00:02.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:02 smithi161 ceph-mon[26055]: pgmap v168: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:02.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:02 smithi161 ceph-mon[26055]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:00:02.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:02 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:02.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:02 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:02.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:02 smithi116 ceph-mon[25082]: pgmap v168: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:02.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:02 smithi116 ceph-mon[25082]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:00:02.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:02 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:02.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:02 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:02.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:02 smithi116 ceph-mon[29958]: pgmap v168: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:02.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:02 smithi116 ceph-mon[29958]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:00:02.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:02 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:02.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:02 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:04.419 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:04 smithi116 ceph-mon[29958]: pgmap v169: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:04.419 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:04 smithi116 ceph-mon[25082]: pgmap v169: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:04.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:04 smithi161 ceph-mon[26055]: pgmap v169: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:04.695 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:00:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:00:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-06-01T02:00:06.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:06 smithi161 ceph-mon[26055]: pgmap v170: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:06.739 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:06 smithi116 ceph-mon[25082]: pgmap v170: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:06.739 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:06 smithi116 ceph-mon[29958]: pgmap v170: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:08.610 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:08 smithi116 ceph-mon[25082]: pgmap v171: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:08.610 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:08 smithi116 ceph-mon[29958]: pgmap v171: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:08.611 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:00:08 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:00:08] "GET /metrics HTTP/1.1" 200 214974 "" "Prometheus/2.33.4" 2024-06-01T02:00:08.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:08 smithi161 ceph-mon[26055]: pgmap v171: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:08.973 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-06-01T02:00:10.452 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:10 smithi116 ceph-mon[29958]: pgmap v172: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:10.452 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:10 smithi116 ceph-mon[25082]: pgmap v172: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:10.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:10 smithi161 ceph-mon[26055]: pgmap v172: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:11.397 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-06-01T02:00:12.681 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:12 smithi116 ceph-mon[29958]: pgmap v173: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:12.681 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:12 smithi116 ceph-mon[29958]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:00:12.681 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:12 smithi116 ceph-mon[25082]: pgmap v173: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:12.681 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:12 smithi116 ceph-mon[25082]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:00:12.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:12 smithi161 ceph-mon[26055]: pgmap v173: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:12.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:12 smithi161 ceph-mon[26055]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:00:13.956 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph config set global log_to_journald false --force' 2024-06-01T02:00:14.542 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:14 smithi116 ceph-mon[25082]: pgmap v174: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:14.542 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:14 smithi116 ceph-mon[29958]: pgmap v174: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:14.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:14 smithi161 ceph-mon[26055]: pgmap v174: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:14.695 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:00:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:00:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-06-01T02:00:15.212 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:00:15 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:02:00:15] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T02:00:16.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:16 smithi161 ceph-mon[26055]: pgmap v175: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:16.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:16 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:16.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:16 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:16.461 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:16 smithi116 ceph-mon[25082]: pgmap v175: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:16.462 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:16 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:16.462 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:16 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:16.462 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:16 smithi116 ceph-mon[29958]: pgmap v175: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:16.462 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:16 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:16.462 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:16 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:16.484 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-06-01T02:00:18.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:18 smithi116 ceph-mon[25082]: pgmap v176: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:18.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:18 smithi116 ceph-mon[29958]: pgmap v176: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:18.320 INFO:teuthology.orchestra.run.smithi116.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:00:18.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:18 smithi161 ceph-mon[26055]: pgmap v176: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:18.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:00:18 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:00:18] "GET /metrics HTTP/1.1" 200 214974 "" "Prometheus/2.33.4" 2024-06-01T02:00:18.874 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-01T02:00:18.883 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi116.front.sepia.ceph.com 2024-06-01T02:00:18.883 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-06-01T02:00:19.452 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:19 smithi116 ceph-mon[29958]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:00:19.453 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:19 smithi116 ceph-mon[29958]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:00:19.453 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:19 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:19.453 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:19 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:00:19.453 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:19 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:00:19.453 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:19 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:19.453 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:19 smithi116 ceph-mon[29958]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:00:19.454 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:19 smithi116 ceph-mon[25082]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:00:19.454 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:19 smithi116 ceph-mon[25082]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:00:19.454 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:19 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:19.454 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:19 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:00:19.454 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:19 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:00:19.455 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:19 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:19.455 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:19 smithi116 ceph-mon[25082]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:00:19.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:19 smithi161 ceph-mon[26055]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:00:19.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:19 smithi161 ceph-mon[26055]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:00:19.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:19 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:19.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:19 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:00:19.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:19 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:00:19.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:19 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:19.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:19 smithi161 ceph-mon[26055]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:00:20.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:20 smithi116 ceph-mon[29958]: pgmap v177: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:20.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:20 smithi116 ceph-mon[25082]: pgmap v177: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:20.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:20 smithi161 ceph-mon[26055]: pgmap v177: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:21.101 INFO:teuthology.orchestra.run.smithi116.stdout:true 2024-06-01T02:00:21.936 INFO:teuthology.orchestra.run.smithi116.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-01T02:00:21.937 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi116 smithi116 running 19s ago 9m - - 2024-06-01T02:00:21.937 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi161 smithi161 running 6s ago 8m - - 2024-06-01T02:00:21.937 INFO:teuthology.orchestra.run.smithi116.stdout:alertmanager.a smithi116 *:9093,9094 running (3m) 19s ago 3m 27.7M - ba2b418f427c 7cd3733dd584 2024-06-01T02:00:21.937 INFO:teuthology.orchestra.run.smithi116.stdout:grafana.a smithi161 *:3000 running (3m) 6s ago 3m 53.8M - 8.3.5 dad864ee21e9 f4be12ac83e0 2024-06-01T02:00:21.937 INFO:teuthology.orchestra.run.smithi116.stdout:iscsi.foo.smithi116.gjuxzb smithi116 running (2m) 19s ago 2m 48.2M - 3.5 e1d6a67b021e 5b3b199ca2db 2024-06-01T02:00:21.937 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.x smithi161 *:8443 running (8m) 6s ago 8m 411M - 17.2.0 e1d6a67b021e f31540b96338 2024-06-01T02:00:21.937 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.y smithi116 *:9283 running (9m) 19s ago 9m 461M - 17.2.0 e1d6a67b021e 3c1beb78ef72 2024-06-01T02:00:21.937 INFO:teuthology.orchestra.run.smithi116.stdout:mon.a smithi116 running (9m) 19s ago 9m 49.6M 2048M 17.2.0 e1d6a67b021e c821092a538a 2024-06-01T02:00:21.937 INFO:teuthology.orchestra.run.smithi116.stdout:mon.b smithi161 running (8m) 6s ago 8m 41.0M 2048M 17.2.0 e1d6a67b021e 9011986aa6e3 2024-06-01T02:00:21.937 INFO:teuthology.orchestra.run.smithi116.stdout:mon.c smithi116 running (8m) 19s ago 8m 42.7M 2048M 17.2.0 e1d6a67b021e 4a6fe0b16d63 2024-06-01T02:00:21.937 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.a smithi116 *:9100 running (4m) 19s ago 4m 15.6M - 1dbe0e931976 35a72a0577d4 2024-06-01T02:00:21.938 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.b smithi161 *:9100 running (4m) 6s ago 4m 16.8M - 1dbe0e931976 4456eaae965b 2024-06-01T02:00:21.938 INFO:teuthology.orchestra.run.smithi116.stdout:osd.0 smithi116 running (7m) 19s ago 7m 52.3M 4096M 17.2.0 e1d6a67b021e 0cc929b0ecad 2024-06-01T02:00:21.938 INFO:teuthology.orchestra.run.smithi116.stdout:osd.1 smithi116 running (7m) 19s ago 7m 53.5M 4096M 17.2.0 e1d6a67b021e 6ff7fc5abe58 2024-06-01T02:00:21.938 INFO:teuthology.orchestra.run.smithi116.stdout:osd.2 smithi116 running (7m) 19s ago 7m 49.4M 4096M 17.2.0 e1d6a67b021e 6aecbb10256b 2024-06-01T02:00:21.938 INFO:teuthology.orchestra.run.smithi116.stdout:osd.3 smithi116 running (6m) 19s ago 6m 52.1M 4096M 17.2.0 e1d6a67b021e a802290f5290 2024-06-01T02:00:21.938 INFO:teuthology.orchestra.run.smithi116.stdout:osd.4 smithi161 running (6m) 6s ago 6m 54.4M 4096M 17.2.0 e1d6a67b021e 70b3f89b0bd9 2024-06-01T02:00:21.938 INFO:teuthology.orchestra.run.smithi116.stdout:osd.5 smithi161 running (5m) 6s ago 5m 49.0M 4096M 17.2.0 e1d6a67b021e 68358b78a25d 2024-06-01T02:00:21.938 INFO:teuthology.orchestra.run.smithi116.stdout:osd.6 smithi161 running (5m) 6s ago 5m 47.8M 4096M 17.2.0 e1d6a67b021e 50abeb0551e7 2024-06-01T02:00:21.938 INFO:teuthology.orchestra.run.smithi116.stdout:osd.7 smithi161 running (5m) 6s ago 5m 52.1M 4096M 17.2.0 e1d6a67b021e 6004e6983735 2024-06-01T02:00:21.938 INFO:teuthology.orchestra.run.smithi116.stdout:prometheus.a smithi161 *:9095 running (2m) 6s ago 4m 64.4M - 514e6a882f6e bea6bf7e7083 2024-06-01T02:00:21.938 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi116.vzjrxj smithi116 *:8000 running (2m) 19s ago 2m 72.0M - 17.2.0 e1d6a67b021e df4fe3c1dbc3 2024-06-01T02:00:21.938 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi161.ashqxz smithi161 *:8000 running (2m) 6s ago 2m 73.2M - 17.2.0 e1d6a67b021e 3eecc1f3e2d3 2024-06-01T02:00:21.938 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi116.hokbmf smithi116 *:80 running (2m) 19s ago 2m 72.8M - 17.2.0 e1d6a67b021e 10950ed75dbd 2024-06-01T02:00:21.939 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi161.axjlzw smithi161 *:80 running (2m) 6s ago 2m 72.0M - 17.2.0 e1d6a67b021e 0b5d5ce47772 2024-06-01T02:00:22.294 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:00:22 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:02:00:22] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T02:00:22.439 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:00:22.440 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": { 2024-06-01T02:00:22.440 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 3 2024-06-01T02:00:22.440 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:00:22.440 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": { 2024-06-01T02:00:22.440 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 2 2024-06-01T02:00:22.440 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:00:22.440 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": { 2024-06-01T02:00:22.440 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2024-06-01T02:00:22.440 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:00:22.440 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": {}, 2024-06-01T02:00:22.440 INFO:teuthology.orchestra.run.smithi116.stdout: "rgw": { 2024-06-01T02:00:22.440 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-06-01T02:00:22.440 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:00:22.441 INFO:teuthology.orchestra.run.smithi116.stdout: "overall": { 2024-06-01T02:00:22.441 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 17 2024-06-01T02:00:22.441 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T02:00:22.441 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:00:22.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:22 smithi161 ceph-mon[26055]: pgmap v178: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:22.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:22 smithi161 ceph-mon[26055]: from='client.24970 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:00:22.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:22 smithi161 ceph-mon[26055]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:00:22.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:22 smithi161 ceph-mon[26055]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:00:22.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:22 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:22.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:22 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:22.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:22 smithi116 ceph-mon[29958]: pgmap v178: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:22.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:22 smithi116 ceph-mon[29958]: from='client.24970 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:00:22.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:22 smithi116 ceph-mon[29958]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:00:22.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:22 smithi116 ceph-mon[29958]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:00:22.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:22 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:22.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:22 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:22.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:22 smithi116 ceph-mon[25082]: pgmap v178: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:22.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:22 smithi116 ceph-mon[25082]: from='client.24970 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:00:22.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:22 smithi116 ceph-mon[25082]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:00:22.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:22 smithi116 ceph-mon[25082]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:00:22.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:22 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:22.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:22 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:22.861 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:00:22.861 INFO:teuthology.orchestra.run.smithi116.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-01T02:00:22.861 INFO:teuthology.orchestra.run.smithi116.stdout: "in_progress": true, 2024-06-01T02:00:22.862 INFO:teuthology.orchestra.run.smithi116.stdout: "services_complete": [], 2024-06-01T02:00:22.862 INFO:teuthology.orchestra.run.smithi116.stdout: "progress": "", 2024-06-01T02:00:22.862 INFO:teuthology.orchestra.run.smithi116.stdout: "message": "Doing first pull of quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba image" 2024-06-01T02:00:22.862 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:00:23.282 INFO:teuthology.orchestra.run.smithi116.stdout:HEALTH_OK 2024-06-01T02:00:23.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:23 smithi116 ceph-mon[25082]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:00:23.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:23 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/2247746313' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:00:23.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:23 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1412344734' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:00:23.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:23 smithi116 ceph-mon[29958]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:00:23.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:23 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/2247746313' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:00:23.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:23 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1412344734' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:00:23.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:23 smithi161 ceph-mon[26055]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:00:23.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:23 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/2247746313' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:00:23.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:23 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1412344734' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:00:24.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:24 smithi161 ceph-mon[26055]: pgmap v179: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:24.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:24 smithi161 ceph-mon[26055]: from='client.24994 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:00:24.695 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:00:24 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:00:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-06-01T02:00:24.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:24 smithi116 ceph-mon[25082]: pgmap v179: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:24.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:24 smithi116 ceph-mon[25082]: from='client.24994 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:00:24.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:24 smithi116 ceph-mon[29958]: pgmap v179: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:24.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:24 smithi116 ceph-mon[29958]: from='client.24994 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:00:26.565 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:26 smithi116 ceph-mon[29958]: pgmap v180: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:26.565 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:26 smithi116 ceph-mon[25082]: pgmap v180: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:26.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:26 smithi161 ceph-mon[26055]: pgmap v180: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:28.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:28 smithi161 ceph-mon[26055]: pgmap v181: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:28.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:28 smithi116 ceph-mon[29958]: pgmap v181: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:28.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:28 smithi116 ceph-mon[25082]: pgmap v181: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:28.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:00:28 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:00:28] "GET /metrics HTTP/1.1" 200 214961 "" "Prometheus/2.33.4" 2024-06-01T02:00:30.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:30 smithi161 ceph-mon[26055]: pgmap v182: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:30.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:30 smithi116 ceph-mon[25082]: pgmap v182: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:30.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:30 smithi116 ceph-mon[29958]: pgmap v182: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:32.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:32 smithi161 ceph-mon[26055]: pgmap v183: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:32.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:32 smithi161 ceph-mon[26055]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:00:32.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:32 smithi116 ceph-mon[29958]: pgmap v183: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:32.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:32 smithi116 ceph-mon[29958]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:00:32.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:32 smithi116 ceph-mon[25082]: pgmap v183: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:32.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:32 smithi116 ceph-mon[25082]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:00:34.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:34 smithi161 ceph-mon[26055]: pgmap v184: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:34.695 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:00:34 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:00:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-06-01T02:00:34.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:34 smithi116 ceph-mon[25082]: pgmap v184: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:34.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:34 smithi116 ceph-mon[29958]: pgmap v184: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:35.544 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:00:35 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:02:00:35] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T02:00:36.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:36 smithi161 ceph-mon[26055]: pgmap v185: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:36.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:36 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:36.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:36 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:36.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:36 smithi116 ceph-mon[29958]: pgmap v185: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:36.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:36 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:36.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:36 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:36.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:36 smithi116 ceph-mon[25082]: pgmap v185: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:36.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:36 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:36.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:36 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:38.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:38 smithi161 ceph-mon[26055]: pgmap v186: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:38.526 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:38 smithi116 ceph-mon[25082]: pgmap v186: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:38.526 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:38 smithi116 ceph-mon[29958]: pgmap v186: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:38.793 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:00:38 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:00:38] "GET /metrics HTTP/1.1" 200 214961 "" "Prometheus/2.33.4" 2024-06-01T02:00:40.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:40 smithi161 ceph-mon[26055]: pgmap v187: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:40.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:40 smithi116 ceph-mon[25082]: pgmap v187: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:40.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:40 smithi116 ceph-mon[29958]: pgmap v187: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:42.543 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:00:42 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:02:00:42] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T02:00:42.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:42 smithi116 ceph-mon[25082]: pgmap v188: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:42.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:42 smithi116 ceph-mon[25082]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:00:42.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:42 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:42.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:42 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:42.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:42 smithi116 ceph-mon[29958]: pgmap v188: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:42.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:42 smithi116 ceph-mon[29958]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:00:42.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:42 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:42.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:42 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:42 smithi161 ceph-mon[26055]: pgmap v188: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:42 smithi161 ceph-mon[26055]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:00:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:42 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:42 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:43.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T02:00:43.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:43 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T02:00:43.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T02:00:43.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:43 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T02:00:43.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T02:00:43.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:43 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T02:00:44.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:44 smithi161 ceph-mon[26055]: pgmap v189: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:44.695 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:00:44 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:00:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-06-01T02:00:44.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:44 smithi116 ceph-mon[25082]: pgmap v189: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:44.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:44 smithi116 ceph-mon[29958]: pgmap v189: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:46.596 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:46 smithi116 ceph-mon[25082]: pgmap v190: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:46.596 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:46 smithi116 ceph-mon[29958]: pgmap v190: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:46.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:46 smithi161 ceph-mon[26055]: pgmap v190: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:48.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:48 smithi161 ceph-mon[26055]: pgmap v191: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:48.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:48 smithi116 ceph-mon[25082]: pgmap v191: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:48.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:48 smithi116 ceph-mon[29958]: pgmap v191: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:48.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:00:48 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:00:48] "GET /metrics HTTP/1.1" 200 214946 "" "Prometheus/2.33.4" 2024-06-01T02:00:50.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:50 smithi161 ceph-mon[26055]: pgmap v192: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:50.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:50 smithi116 ceph-mon[25082]: pgmap v192: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:50.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:50 smithi116 ceph-mon[29958]: pgmap v192: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:52.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:52 smithi116 ceph-mon[25082]: pgmap v193: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:52.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:52 smithi116 ceph-mon[25082]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:00:52.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:52 smithi116 ceph-mon[29958]: pgmap v193: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:52.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:52 smithi116 ceph-mon[29958]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:00:52.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:52 smithi161 ceph-mon[26055]: pgmap v193: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:52.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:52 smithi161 ceph-mon[26055]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:00:53.610 INFO:teuthology.orchestra.run.smithi116.stdout:true 2024-06-01T02:00:53.739 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:00:53 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=warn ts=2024-06-01T02:00:53.508Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.161:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.161 because it doesn't contain any IP SANs" 2024-06-01T02:00:53.739 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:00:53 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=warn ts=2024-06-01T02:00:53.508Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.116:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.116 because it doesn't contain any IP SANs" 2024-06-01T02:00:54.216 INFO:teuthology.orchestra.run.smithi116.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-01T02:00:54.216 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi116 smithi116 running 11s ago 9m - - 2024-06-01T02:00:54.216 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi161 smithi161 running 19s ago 9m - - 2024-06-01T02:00:54.216 INFO:teuthology.orchestra.run.smithi116.stdout:alertmanager.a smithi116 *:9093,9094 running (3m) 11s ago 4m 26.4M - ba2b418f427c 7cd3733dd584 2024-06-01T02:00:54.216 INFO:teuthology.orchestra.run.smithi116.stdout:grafana.a smithi161 *:3000 running (3m) 19s ago 3m 53.8M - 8.3.5 dad864ee21e9 f4be12ac83e0 2024-06-01T02:00:54.216 INFO:teuthology.orchestra.run.smithi116.stdout:iscsi.foo.smithi116.gjuxzb smithi116 running (2m) 11s ago 2m 48.4M - 3.5 e1d6a67b021e 5b3b199ca2db 2024-06-01T02:00:54.216 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.x smithi161 *:8443 running (8m) 19s ago 8m 412M - 17.2.0 e1d6a67b021e f31540b96338 2024-06-01T02:00:54.217 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.y smithi116 *:9283 running (10m) 11s ago 10m 462M - 17.2.0 e1d6a67b021e 3c1beb78ef72 2024-06-01T02:00:54.217 INFO:teuthology.orchestra.run.smithi116.stdout:mon.a smithi116 running (10m) 11s ago 10m 52.3M 2048M 17.2.0 e1d6a67b021e c821092a538a 2024-06-01T02:00:54.217 INFO:teuthology.orchestra.run.smithi116.stdout:mon.b smithi161 running (9m) 19s ago 8m 36.9M 2048M 17.2.0 e1d6a67b021e 9011986aa6e3 2024-06-01T02:00:54.217 INFO:teuthology.orchestra.run.smithi116.stdout:mon.c smithi116 running (9m) 11s ago 9m 44.5M 2048M 17.2.0 e1d6a67b021e 4a6fe0b16d63 2024-06-01T02:00:54.217 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.a smithi116 *:9100 running (4m) 11s ago 4m 15.8M - 1dbe0e931976 35a72a0577d4 2024-06-01T02:00:54.217 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.b smithi161 *:9100 running (4m) 19s ago 4m 16.5M - 1dbe0e931976 4456eaae965b 2024-06-01T02:00:54.217 INFO:teuthology.orchestra.run.smithi116.stdout:osd.0 smithi116 running (8m) 11s ago 8m 52.7M 4096M 17.2.0 e1d6a67b021e 0cc929b0ecad 2024-06-01T02:00:54.217 INFO:teuthology.orchestra.run.smithi116.stdout:osd.1 smithi116 running (8m) 11s ago 8m 53.8M 4096M 17.2.0 e1d6a67b021e 6ff7fc5abe58 2024-06-01T02:00:54.217 INFO:teuthology.orchestra.run.smithi116.stdout:osd.2 smithi116 running (7m) 11s ago 7m 49.7M 4096M 17.2.0 e1d6a67b021e 6aecbb10256b 2024-06-01T02:00:54.217 INFO:teuthology.orchestra.run.smithi116.stdout:osd.3 smithi116 running (7m) 11s ago 7m 52.4M 4096M 17.2.0 e1d6a67b021e a802290f5290 2024-06-01T02:00:54.218 INFO:teuthology.orchestra.run.smithi116.stdout:osd.4 smithi161 running (6m) 19s ago 6m 54.6M 4096M 17.2.0 e1d6a67b021e 70b3f89b0bd9 2024-06-01T02:00:54.218 INFO:teuthology.orchestra.run.smithi116.stdout:osd.5 smithi161 running (6m) 19s ago 6m 49.2M 4096M 17.2.0 e1d6a67b021e 68358b78a25d 2024-06-01T02:00:54.218 INFO:teuthology.orchestra.run.smithi116.stdout:osd.6 smithi161 running (6m) 19s ago 6m 47.9M 4096M 17.2.0 e1d6a67b021e 50abeb0551e7 2024-06-01T02:00:54.218 INFO:teuthology.orchestra.run.smithi116.stdout:osd.7 smithi161 running (5m) 19s ago 5m 52.2M 4096M 17.2.0 e1d6a67b021e 6004e6983735 2024-06-01T02:00:54.218 INFO:teuthology.orchestra.run.smithi116.stdout:prometheus.a smithi161 *:9095 running (3m) 19s ago 4m 66.6M - 514e6a882f6e bea6bf7e7083 2024-06-01T02:00:54.218 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi116.vzjrxj smithi116 *:8000 running (3m) 11s ago 3m 72.3M - 17.2.0 e1d6a67b021e df4fe3c1dbc3 2024-06-01T02:00:54.218 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi161.ashqxz smithi161 *:8000 running (3m) 19s ago 3m 73.3M - 17.2.0 e1d6a67b021e 3eecc1f3e2d3 2024-06-01T02:00:54.218 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi116.hokbmf smithi116 *:80 running (2m) 11s ago 2m 73.1M - 17.2.0 e1d6a67b021e 10950ed75dbd 2024-06-01T02:00:54.218 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi161.axjlzw smithi161 *:80 running (2m) 19s ago 2m 72.1M - 17.2.0 e1d6a67b021e 0b5d5ce47772 2024-06-01T02:00:54.581 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:00:54.581 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": { 2024-06-01T02:00:54.581 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 3 2024-06-01T02:00:54.581 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:00:54.581 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": { 2024-06-01T02:00:54.581 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 2 2024-06-01T02:00:54.581 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:00:54.582 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": { 2024-06-01T02:00:54.582 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2024-06-01T02:00:54.582 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:00:54.582 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": {}, 2024-06-01T02:00:54.582 INFO:teuthology.orchestra.run.smithi116.stdout: "rgw": { 2024-06-01T02:00:54.582 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-06-01T02:00:54.582 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:00:54.582 INFO:teuthology.orchestra.run.smithi116.stdout: "overall": { 2024-06-01T02:00:54.582 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 17 2024-06-01T02:00:54.582 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T02:00:54.582 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:00:54.695 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:00:54 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:00:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-06-01T02:00:54.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:54 smithi161 ceph-mon[26055]: pgmap v194: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:54.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:54 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:54.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:54 smithi161 ceph-mon[26055]: Upgrade: Target is version 19.0.0-2946-g3d89accf (unknown) 2024-06-01T02:00:54.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:54 smithi161 ceph-mon[26055]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, digests ['quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9'] 2024-06-01T02:00:54.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:54 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:00:54.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:54 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:54.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:54 smithi161 ceph-mon[26055]: Upgrade: Need to upgrade myself (mgr.y) 2024-06-01T02:00:54.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:54 smithi161 ceph-mon[26055]: from='client.25006 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:00:54.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:54 smithi116 ceph-mon[25082]: pgmap v194: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:54.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:54 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:54.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:54 smithi116 ceph-mon[25082]: Upgrade: Target is version 19.0.0-2946-g3d89accf (unknown) 2024-06-01T02:00:54.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:54 smithi116 ceph-mon[25082]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, digests ['quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9'] 2024-06-01T02:00:54.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:54 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:00:54.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:54 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:54.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:54 smithi116 ceph-mon[25082]: Upgrade: Need to upgrade myself (mgr.y) 2024-06-01T02:00:54.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:54 smithi116 ceph-mon[25082]: from='client.25006 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:00:54.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:54 smithi116 ceph-mon[29958]: pgmap v194: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:54.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:54 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:54.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:54 smithi116 ceph-mon[29958]: Upgrade: Target is version 19.0.0-2946-g3d89accf (unknown) 2024-06-01T02:00:54.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:54 smithi116 ceph-mon[29958]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, digests ['quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9'] 2024-06-01T02:00:54.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:54 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:00:54.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:54 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:54.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:54 smithi116 ceph-mon[29958]: Upgrade: Need to upgrade myself (mgr.y) 2024-06-01T02:00:54.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:54 smithi116 ceph-mon[29958]: from='client.25006 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:00:54.882 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:00:54.882 INFO:teuthology.orchestra.run.smithi116.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-01T02:00:54.882 INFO:teuthology.orchestra.run.smithi116.stdout: "in_progress": true, 2024-06-01T02:00:54.882 INFO:teuthology.orchestra.run.smithi116.stdout: "services_complete": [], 2024-06-01T02:00:54.883 INFO:teuthology.orchestra.run.smithi116.stdout: "progress": "0/23 daemons upgraded", 2024-06-01T02:00:54.883 INFO:teuthology.orchestra.run.smithi116.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba image on host smithi161" 2024-06-01T02:00:54.883 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:00:55.232 INFO:teuthology.orchestra.run.smithi116.stdout:HEALTH_OK 2024-06-01T02:00:55.497 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:00:55 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:02:00:55] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T02:00:55.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:55 smithi161 ceph-mon[26055]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba on smithi161 2024-06-01T02:00:55.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:55 smithi161 ceph-mon[26055]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:00:55.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:55 smithi161 ceph-mon[26055]: from='client.25018 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:00:55.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:55 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/2699755875' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:00:55.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:55 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/302018269' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:00:55.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:55 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:55.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:55 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:55.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:55 smithi116 ceph-mon[29958]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba on smithi161 2024-06-01T02:00:55.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:55 smithi116 ceph-mon[29958]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:00:55.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:55 smithi116 ceph-mon[29958]: from='client.25018 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:00:55.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:55 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/2699755875' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:00:55.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:55 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/302018269' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:00:55.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:55 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:55.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:55 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:55.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:55 smithi116 ceph-mon[25082]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba on smithi161 2024-06-01T02:00:55.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:55 smithi116 ceph-mon[25082]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:00:55.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:55 smithi116 ceph-mon[25082]: from='client.25018 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:00:55.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:55 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/2699755875' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:00:55.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:55 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/302018269' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:00:55.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:55 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:55.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:55 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:00:56.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:56 smithi116 ceph-mon[25082]: pgmap v195: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:56.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:56 smithi116 ceph-mon[25082]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:00:56.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:56 smithi116 ceph-mon[29958]: pgmap v195: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:56.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:56 smithi116 ceph-mon[29958]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:00:56.816 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:56 smithi161 ceph-mon[26055]: pgmap v195: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:00:56.817 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:56 smithi161 ceph-mon[26055]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:00:58.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:00:58 smithi116 ceph-mon[25082]: pgmap v196: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:58.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:00:58 smithi116 ceph-mon[29958]: pgmap v196: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:00:58.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:00:58 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:00:58] "GET /metrics HTTP/1.1" 200 214923 "" "Prometheus/2.33.4" 2024-06-01T02:00:58.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:00:58 smithi161 ceph-mon[26055]: pgmap v196: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:00.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:00 smithi116 ceph-mon[25082]: pgmap v197: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:01:00.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:00 smithi116 ceph-mon[29958]: pgmap v197: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:01:00.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:00 smithi161 ceph-mon[26055]: pgmap v197: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:01:02.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:02 smithi116 ceph-mon[25082]: pgmap v198: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:02.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:02 smithi116 ceph-mon[25082]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:01:02.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:02 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:02.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:02 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:02.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:02:01:02] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T02:01:02.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:02 smithi116 ceph-mon[29958]: pgmap v198: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:02.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:02 smithi116 ceph-mon[29958]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:01:02.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:02 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:02.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:02 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:02.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:02 smithi161 ceph-mon[26055]: pgmap v198: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:02.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:02 smithi161 ceph-mon[26055]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:01:02.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:02 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:02.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:02 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:03.794 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:01:03 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=error ts=2024-06-01T02:01:03.506Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=6 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.116:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.116 because it doesn't contain any IP SANs; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://172.21.15.161:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.161 because it doesn't contain any IP SANs" 2024-06-01T02:01:03.794 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:01:03 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=warn ts=2024-06-01T02:01:03.508Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.116:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.116 because it doesn't contain any IP SANs" 2024-06-01T02:01:03.794 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:01:03 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=warn ts=2024-06-01T02:01:03.509Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.161:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.161 because it doesn't contain any IP SANs" 2024-06-01T02:01:04.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:04 smithi161 ceph-mon[26055]: pgmap v199: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:04.695 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:01:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-06-01T02:01:04.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:04 smithi116 ceph-mon[25082]: pgmap v199: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:04.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:04 smithi116 ceph-mon[29958]: pgmap v199: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:06.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:06 smithi116 ceph-mon[25082]: pgmap v200: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:01:06.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:06 smithi116 ceph-mon[29958]: pgmap v200: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:01:06.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:06 smithi161 ceph-mon[26055]: pgmap v200: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:01:08.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:08 smithi116 ceph-mon[29958]: pgmap v201: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:08.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:08 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:01:08] "GET /metrics HTTP/1.1" 200 214923 "" "Prometheus/2.33.4" 2024-06-01T02:01:08.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:08 smithi116 ceph-mon[25082]: pgmap v201: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:08.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:08 smithi161 ceph-mon[26055]: pgmap v201: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:10.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:10 smithi116 ceph-mon[25082]: pgmap v202: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:01:10.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:10 smithi116 ceph-mon[29958]: pgmap v202: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:01:10.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:10 smithi161 ceph-mon[26055]: pgmap v202: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:01:12.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:12 smithi116 ceph-mon[25082]: pgmap v203: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:12.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:12 smithi116 ceph-mon[25082]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:01:12.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:12 smithi116 ceph-mon[29958]: pgmap v203: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:12.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:12 smithi116 ceph-mon[29958]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:01:12.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:12 smithi161 ceph-mon[26055]: pgmap v203: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:12.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:12 smithi161 ceph-mon[26055]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:01:13.793 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:01:13 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=error ts=2024-06-01T02:01:13.508Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=6 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.161:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.161 because it doesn't contain any IP SANs; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://172.21.15.116:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.116 because it doesn't contain any IP SANs" 2024-06-01T02:01:13.794 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:01:13 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=warn ts=2024-06-01T02:01:13.509Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.116:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.116 because it doesn't contain any IP SANs" 2024-06-01T02:01:13.794 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:01:13 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=warn ts=2024-06-01T02:01:13.510Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.161:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.161 because it doesn't contain any IP SANs" 2024-06-01T02:01:14.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:14 smithi161 ceph-mon[26055]: pgmap v204: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:14.695 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:01:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-06-01T02:01:14.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:14 smithi116 ceph-mon[29958]: pgmap v204: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:14.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:14 smithi116 ceph-mon[25082]: pgmap v204: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:15.793 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:15 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:02:01:15] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T02:01:16.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:16 smithi161 ceph-mon[26055]: pgmap v205: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:01:16.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:16 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:16.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:16 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:16.695 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:16 smithi116 ceph-mon[25082]: pgmap v205: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:01:16.695 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:16 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:16.695 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:16 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:16.696 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:16 smithi116 ceph-mon[29958]: pgmap v205: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:01:16.696 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:16 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:16.696 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:16 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:18.365 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:18 smithi116 ceph-mon[25082]: pgmap v206: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:18.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:18 smithi161 ceph-mon[26055]: pgmap v206: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:18.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:18 smithi116 ceph-mon[29958]: pgmap v206: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:18.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:18 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:01:18] "GET /metrics HTTP/1.1" 200 214916 "" "Prometheus/2.33.4" 2024-06-01T02:01:20.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:20 smithi161 ceph-mon[26055]: pgmap v207: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:01:20.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:20 smithi116 ceph-mon[25082]: pgmap v207: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:01:20.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:20 smithi116 ceph-mon[29958]: pgmap v207: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:01:22.373 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:22 smithi161 ceph-mon[26055]: pgmap v208: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:22.374 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:22 smithi161 ceph-mon[26055]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:01:22.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:22 smithi116 ceph-mon[29958]: pgmap v208: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:22.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:22 smithi116 ceph-mon[29958]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:01:22.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:22 smithi116 ceph-mon[25082]: pgmap v208: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:22.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:22 smithi116 ceph-mon[25082]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:01:22.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:22 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.116 - - [01/Jun/2024:02:01:22] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T02:01:23.678 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:23 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:23.678 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:23 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:23.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:23 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:23.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:23 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:23.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:23 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:23.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:23 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:23.795 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:01:23 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=error ts=2024-06-01T02:01:23.509Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=6 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.161:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.161 because it doesn't contain any IP SANs; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://172.21.15.116:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.116 because it doesn't contain any IP SANs" 2024-06-01T02:01:23.795 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:01:23 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=warn ts=2024-06-01T02:01:23.510Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.116:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.116 because it doesn't contain any IP SANs" 2024-06-01T02:01:23.795 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:01:23 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=warn ts=2024-06-01T02:01:23.511Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.161:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.161 because it doesn't contain any IP SANs" 2024-06-01T02:01:24.628 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:24 smithi161 ceph-mon[26055]: pgmap v209: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:24.628 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:24 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[27600]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:01:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-06-01T02:01:24.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:24 smithi116 ceph-mon[25082]: pgmap v209: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:24.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:24 smithi116 ceph-mon[29958]: pgmap v209: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:25.563 INFO:teuthology.orchestra.run.smithi116.stdout:true 2024-06-01T02:01:26.153 INFO:teuthology.orchestra.run.smithi116.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-01T02:01:26.153 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi116 smithi116 running 3s ago 10m - - 2024-06-01T02:01:26.153 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi161 smithi161 running 10s ago 9m - - 2024-06-01T02:01:26.153 INFO:teuthology.orchestra.run.smithi116.stdout:alertmanager.a smithi116 *:9093,9094 running (4m) 3s ago 4m 26.5M - ba2b418f427c 7cd3733dd584 2024-06-01T02:01:26.153 INFO:teuthology.orchestra.run.smithi116.stdout:grafana.a smithi161 *:3000 running (4m) 10s ago 4m 52.7M - 8.3.5 dad864ee21e9 f4be12ac83e0 2024-06-01T02:01:26.153 INFO:teuthology.orchestra.run.smithi116.stdout:iscsi.foo.smithi116.gjuxzb smithi116 running (3m) 3s ago 3m 48.5M - 3.5 e1d6a67b021e 5b3b199ca2db 2024-06-01T02:01:26.153 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.x smithi161 *:8443 running (9m) 10s ago 9m 412M - 17.2.0 e1d6a67b021e f31540b96338 2024-06-01T02:01:26.153 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.y smithi116 *:9283 running (10m) 3s ago 10m 463M - 17.2.0 e1d6a67b021e 3c1beb78ef72 2024-06-01T02:01:26.153 INFO:teuthology.orchestra.run.smithi116.stdout:mon.a smithi116 running (10m) 3s ago 11m 51.7M 2048M 17.2.0 e1d6a67b021e c821092a538a 2024-06-01T02:01:26.153 INFO:teuthology.orchestra.run.smithi116.stdout:mon.b smithi161 running (9m) 10s ago 9m 38.3M 2048M 17.2.0 e1d6a67b021e 9011986aa6e3 2024-06-01T02:01:26.153 INFO:teuthology.orchestra.run.smithi116.stdout:mon.c smithi116 running (9m) 3s ago 9m 44.1M 2048M 17.2.0 e1d6a67b021e 4a6fe0b16d63 2024-06-01T02:01:26.154 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.a smithi116 *:9100 running (5m) 3s ago 5m 15.9M - 1dbe0e931976 35a72a0577d4 2024-06-01T02:01:26.154 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.b smithi161 *:9100 running (5m) 10s ago 5m 17.0M - 1dbe0e931976 4456eaae965b 2024-06-01T02:01:26.154 INFO:teuthology.orchestra.run.smithi116.stdout:osd.0 smithi116 running (8m) 3s ago 8m 53.1M 4096M 17.2.0 e1d6a67b021e 0cc929b0ecad 2024-06-01T02:01:26.154 INFO:teuthology.orchestra.run.smithi116.stdout:osd.1 smithi116 running (8m) 3s ago 8m 54.2M 4096M 17.2.0 e1d6a67b021e 6ff7fc5abe58 2024-06-01T02:01:26.154 INFO:teuthology.orchestra.run.smithi116.stdout:osd.2 smithi116 running (8m) 3s ago 8m 49.9M 4096M 17.2.0 e1d6a67b021e 6aecbb10256b 2024-06-01T02:01:26.154 INFO:teuthology.orchestra.run.smithi116.stdout:osd.3 smithi116 running (7m) 3s ago 7m 52.9M 4096M 17.2.0 e1d6a67b021e a802290f5290 2024-06-01T02:01:26.154 INFO:teuthology.orchestra.run.smithi116.stdout:osd.4 smithi161 running (7m) 10s ago 7m 55.1M 4096M 17.2.0 e1d6a67b021e 70b3f89b0bd9 2024-06-01T02:01:26.154 INFO:teuthology.orchestra.run.smithi116.stdout:osd.5 smithi161 running (6m) 10s ago 6m 49.4M 4096M 17.2.0 e1d6a67b021e 68358b78a25d 2024-06-01T02:01:26.154 INFO:teuthology.orchestra.run.smithi116.stdout:osd.6 smithi161 running (6m) 10s ago 6m 48.2M 4096M 17.2.0 e1d6a67b021e 50abeb0551e7 2024-06-01T02:01:26.154 INFO:teuthology.orchestra.run.smithi116.stdout:osd.7 smithi161 running (6m) 10s ago 6m 52.5M 4096M 17.2.0 e1d6a67b021e 6004e6983735 2024-06-01T02:01:26.154 INFO:teuthology.orchestra.run.smithi116.stdout:prometheus.a smithi161 *:9095 running (4m) 10s ago 5m 66.6M - 514e6a882f6e bea6bf7e7083 2024-06-01T02:01:26.154 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi116.vzjrxj smithi116 *:8000 running (3m) 3s ago 3m 72.5M - 17.2.0 e1d6a67b021e df4fe3c1dbc3 2024-06-01T02:01:26.154 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi161.ashqxz smithi161 *:8000 running (3m) 10s ago 3m 73.6M - 17.2.0 e1d6a67b021e 3eecc1f3e2d3 2024-06-01T02:01:26.155 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi116.hokbmf smithi116 *:80 running (3m) 3s ago 3m 73.4M - 17.2.0 e1d6a67b021e 10950ed75dbd 2024-06-01T02:01:26.155 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi161.axjlzw smithi161 *:80 running (3m) 10s ago 3m 72.4M - 17.2.0 e1d6a67b021e 0b5d5ce47772 2024-06-01T02:01:26.507 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:01:26.508 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": { 2024-06-01T02:01:26.508 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 3 2024-06-01T02:01:26.508 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:01:26.508 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": { 2024-06-01T02:01:26.508 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 2 2024-06-01T02:01:26.508 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:01:26.508 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": { 2024-06-01T02:01:26.508 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2024-06-01T02:01:26.508 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:01:26.508 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": {}, 2024-06-01T02:01:26.508 INFO:teuthology.orchestra.run.smithi116.stdout: "rgw": { 2024-06-01T02:01:26.508 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-06-01T02:01:26.509 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:01:26.509 INFO:teuthology.orchestra.run.smithi116.stdout: "overall": { 2024-06-01T02:01:26.509 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 17 2024-06-01T02:01:26.509 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T02:01:26.509 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:01:26.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:26 smithi116 ceph-mon[29958]: pgmap v210: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:01:26.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:26 smithi116 ceph-mon[29958]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:01:26.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:26 smithi116 ceph-mon[25082]: pgmap v210: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:01:26.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:26 smithi116 ceph-mon[25082]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:01:26.807 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:01:26.808 INFO:teuthology.orchestra.run.smithi116.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-01T02:01:26.808 INFO:teuthology.orchestra.run.smithi116.stdout: "in_progress": true, 2024-06-01T02:01:26.808 INFO:teuthology.orchestra.run.smithi116.stdout: "services_complete": [], 2024-06-01T02:01:26.808 INFO:teuthology.orchestra.run.smithi116.stdout: "progress": "0/23 daemons upgraded", 2024-06-01T02:01:26.808 INFO:teuthology.orchestra.run.smithi116.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba image on host smithi161" 2024-06-01T02:01:26.808 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:01:26.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:26 smithi161 ceph-mon[26055]: pgmap v210: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:01:26.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:26 smithi161 ceph-mon[26055]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:01:27.157 INFO:teuthology.orchestra.run.smithi116.stdout:HEALTH_OK 2024-06-01T02:01:27.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:27 smithi161 ceph-mon[26055]: from='client.25048 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:01:27.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:27 smithi161 ceph-mon[26055]: from='client.25054 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:01:27.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:27 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/2557386639' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:01:27.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:27 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/2049504667' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:01:27.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:27 smithi116 ceph-mon[29958]: from='client.25048 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:01:27.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:27 smithi116 ceph-mon[29958]: from='client.25054 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:01:27.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:27 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/2557386639' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:01:27.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:27 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/2049504667' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:01:27.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:27 smithi116 ceph-mon[25082]: from='client.25048 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:01:27.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:27 smithi116 ceph-mon[25082]: from='client.25054 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:01:27.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:27 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/2557386639' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:01:27.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:27 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/2049504667' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:01:28.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:28 smithi116 ceph-mon[25082]: from='client.25066 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:01:28.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:28 smithi116 ceph-mon[25082]: pgmap v211: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:28.795 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:28 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:01:28] "GET /metrics HTTP/1.1" 200 214921 "" "Prometheus/2.33.4" 2024-06-01T02:01:28.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:28 smithi116 ceph-mon[29958]: from='client.25066 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:01:28.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:28 smithi116 ceph-mon[29958]: pgmap v211: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:28.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:28 smithi161 ceph-mon[26055]: from='client.25066 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:01:28.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:28 smithi161 ceph-mon[26055]: pgmap v211: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:30.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:30 smithi116 ceph-mon[25082]: pgmap v212: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:01:30.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:30 smithi116 ceph-mon[29958]: pgmap v212: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:01:30.748 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:30 smithi161 ceph-mon[26055]: pgmap v212: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:01:32.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:32 smithi116 ceph-mon[25082]: pgmap v213: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:32.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:32 smithi116 ceph-mon[25082]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:01:32.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:32 smithi116 ceph-mon[29958]: pgmap v213: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:32.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:32 smithi116 ceph-mon[29958]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:01:32.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:32 smithi161 ceph-mon[26055]: pgmap v213: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:32.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:32 smithi161 ceph-mon[26055]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:01:33.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:33 smithi116 ceph-mon[25082]: Upgrade: Updating mgr.x 2024-06-01T02:01:33.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:33 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:33.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:33 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:01:33.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:33 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-01T02:01:33.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:33 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:01:33.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:33 smithi116 ceph-mon[25082]: Deploying daemon mgr.x on smithi161 2024-06-01T02:01:33.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:33 smithi116 ceph-mon[29958]: Upgrade: Updating mgr.x 2024-06-01T02:01:33.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:33 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:33.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:33 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:01:33.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:33 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-01T02:01:33.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:33 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:01:33.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:33 smithi116 ceph-mon[29958]: Deploying daemon mgr.x on smithi161 2024-06-01T02:01:33.795 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:01:33 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=error ts=2024-06-01T02:01:33.509Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=6 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.161:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.161 because it doesn't contain any IP SANs; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://172.21.15.116:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.116 because it doesn't contain any IP SANs" 2024-06-01T02:01:33.795 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:01:33 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=warn ts=2024-06-01T02:01:33.511Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.116:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.116 because it doesn't contain any IP SANs" 2024-06-01T02:01:33.795 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:01:33 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=warn ts=2024-06-01T02:01:33.511Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.161:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.161 because it doesn't contain any IP SANs" 2024-06-01T02:01:33.885 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:33 smithi161 ceph-mon[26055]: Upgrade: Updating mgr.x 2024-06-01T02:01:33.885 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:33 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:33.885 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:33 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:01:33.885 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:33 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-01T02:01:33.885 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:33 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:01:33.885 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:33 smithi161 ceph-mon[26055]: Deploying daemon mgr.x on smithi161 2024-06-01T02:01:34.195 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:34 smithi161 systemd[1]: Stopping Ceph mgr.x for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:01:34.195 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:34 smithi161 bash[51837]: Error: no container with name or ID "ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr.x" found: no such container 2024-06-01T02:01:34.580 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:34 smithi161 podman[51848]: 2024-06-01 02:01:34.263513321 +0000 UTC m=+0.125010564 container died f31540b9633899be6dfdb70ccfb8d6d8daa0e75b02e630a4f31f3d5690a380e3 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, vendor=Red Hat, Inc., CEPH_POINT_RELEASE=-17.2.0, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, io.openshift.expose-services=, release=754, vcs-type=git, io.buildah.version=1.19.8, maintainer=Guillaume Abrioux , com.redhat.license_terms=https://centos.org/legal/licensing-policy/, architecture=x86_64, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, GIT_REPO=https://github.com/ceph/ceph-container.git, io.openshift.tags=base centos centos-stream, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, name=centos-stream, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, RELEASE=HEAD, com.redhat.component=centos-stream-container, ceph=True, version=8, GIT_CLEAN=True, GIT_BRANCH=HEAD, distribution-scope=public, build-date=2022-05-03T08:36:31.336870, io.k8s.display-name=CentOS Stream 8) 2024-06-01T02:01:34.580 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:34 smithi161 podman[51848]: 2024-06-01 02:01:34.584190589 +0000 UTC m=+0.445687832 container cleanup f31540b9633899be6dfdb70ccfb8d6d8daa0e75b02e630a4f31f3d5690a380e3 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x, io.buildah.version=1.19.8, io.openshift.tags=base centos centos-stream, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, vcs-type=git, name=centos-stream, GIT_CLEAN=True, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, RELEASE=HEAD, build-date=2022-05-03T08:36:31.336870, distribution-scope=public, version=8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., architecture=x86_64, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, io.k8s.display-name=CentOS Stream 8, ceph=True, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, CEPH_POINT_RELEASE=-17.2.0, maintainer=Guillaume Abrioux , release=754, com.redhat.component=centos-stream-container, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, vendor=Red Hat, Inc., com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, io.openshift.expose-services=, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac) 2024-06-01T02:01:34.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:34 smithi161 ceph-mon[26055]: pgmap v214: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:34.945 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:34 smithi161 bash[51848]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x 2024-06-01T02:01:34.946 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:34 smithi161 podman[51860]: 2024-06-01 02:01:34.752238934 +0000 UTC m=+0.484486528 container remove f31540b9633899be6dfdb70ccfb8d6d8daa0e75b02e630a4f31f3d5690a380e3 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x, architecture=x86_64, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, distribution-scope=public, io.openshift.tags=base centos centos-stream, CEPH_POINT_RELEASE=-17.2.0, GIT_BRANCH=HEAD, release=754, RELEASE=HEAD, name=centos-stream, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, maintainer=Guillaume Abrioux , com.redhat.license_terms=https://centos.org/legal/licensing-policy/, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, vcs-type=git, vendor=Red Hat, Inc., url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, io.k8s.display-name=CentOS Stream 8, version=8, io.openshift.expose-services=, build-date=2022-05-03T08:36:31.336870, io.buildah.version=1.19.8, com.redhat.component=centos-stream-container, ceph=True) 2024-06-01T02:01:34.946 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:34 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-06-01T02:01:34.946 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:34 smithi161 bash[51874]: Error: no container with name or ID "ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr.x" found: no such container 2024-06-01T02:01:34.946 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:34 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mgr.x.service: Failed with result 'exit-code'. 2024-06-01T02:01:34.946 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:34 smithi161 systemd[1]: Stopped Ceph mgr.x for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:01:34.946 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:34 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mgr.x.service: Consumed 17.292s CPU time. 2024-06-01T02:01:35.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:34 smithi116 ceph-mon[25082]: pgmap v214: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:35.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:34 smithi116 ceph-mon[29958]: pgmap v214: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:01:35.265 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:35 smithi161 systemd[1]: Starting Ceph mgr.x for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:01:35.585 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:35 smithi161 podman[51977]: 2024-06-01 02:01:35.171358608 +0000 UTC m=+0.017184621 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:01:35.585 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:35 smithi161 podman[51977]: 2024-06-01 02:01:35.269839189 +0000 UTC m=+0.115665185 container create 2743006903a699506c0f492ca6fdfa50e12b615367d268f02173980e72eca60f (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528) 2024-06-01T02:01:35.585 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:35 smithi161 podman[51977]: 2024-06-01 02:01:35.491072835 +0000 UTC m=+0.336898845 container init 2743006903a699506c0f492ca6fdfa50e12b615367d268f02173980e72eca60f (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240528) 2024-06-01T02:01:35.585 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:35 smithi161 podman[51977]: 2024-06-01 02:01:35.494018993 +0000 UTC m=+0.339844998 container start 2743006903a699506c0f492ca6fdfa50e12b615367d268f02173980e72eca60f (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, ceph=True) 2024-06-01T02:01:35.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:35 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 172.21.15.161 - - [01/Jun/2024:02:01:35] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-06-01T02:01:35.850 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:35 smithi161 bash[51977]: 2743006903a699506c0f492ca6fdfa50e12b615367d268f02173980e72eca60f 2024-06-01T02:01:35.850 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:35 smithi161 systemd[1]: Started Ceph mgr.x for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:01:35.850 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:35 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:01:35.655+0000 7fc46c077200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-01T02:01:36.195 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:35 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:01:35.853+0000 7fc46c077200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-01T02:01:36.195 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:35 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:01:35.974+0000 7fc46c077200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-01T02:01:36.293 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:01:35 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=warn ts=2024-06-01T02:01:35.818Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=4 err="Post \"https://172.21.15.161:8443/api/prometheus_receiver\": dial tcp 172.21.15.161:8443: connect: connection refused" 2024-06-01T02:01:36.694 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:36 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:01:36.290+0000 7fc46c077200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-01T02:01:36.695 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:36 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:01:36.379+0000 7fc46c077200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-01T02:01:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:36 smithi161 ceph-mon[26055]: pgmap v215: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:01:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:36 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:36 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:36 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:36 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:01:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:36 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:01:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:36 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:36 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:36 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:36.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:36 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:01:36.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:36 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-06-01T02:01:36.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:36 smithi161 ceph-mon[26055]: osdmap e96: 8 total, 8 up, 8 in 2024-06-01T02:01:36.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:36 smithi116 ceph-mon[25082]: pgmap v215: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:01:36.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:36 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:36.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:36 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:36.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:36 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:36.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:36 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:01:36.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:36 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:01:36.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:36 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:36.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:36 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:36.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:36 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:36.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:36 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:01:36.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:36 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-06-01T02:01:36.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:36 smithi116 ceph-mon[25082]: osdmap e96: 8 total, 8 up, 8 in 2024-06-01T02:01:36.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:36 smithi116 ceph-mon[29958]: pgmap v215: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:01:36.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:36 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:36.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:36 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:36.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:36 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:36.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:36 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:01:36.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:36 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:01:36.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:36 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:36.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:36 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:36.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:36 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' 2024-06-01T02:01:36.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:36 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:01:36.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:36 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-06-01T02:01:36.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:36 smithi116 ceph-mon[29958]: osdmap e96: 8 total, 8 up, 8 in 2024-06-01T02:01:37.238 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:36 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ignoring --setuser ceph since I am not root 2024-06-01T02:01:37.238 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:36 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ignoring --setgroup ceph since I am not root 2024-06-01T02:01:37.238 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:36 smithi116 ceph-mgr[25316]: -- 172.21.15.116:0/2550210945 <== mon.1 v2:172.21.15.116:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557150346340 con 0x55715031ec00 2024-06-01T02:01:37.238 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:37 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T02:01:37.112+0000 7f300a609000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-01T02:01:37.238 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:37 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T02:01:37.240+0000 7f300a609000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-01T02:01:37.543 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:37 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T02:01:37.337+0000 7f300a609000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-01T02:01:37.544 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:01:37 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=warn ts=2024-06-01T02:01:37.434Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=5 err="Post \"https://172.21.15.116:8443//api/prometheus_receiver\": dial tcp 172.21.15.116:8443: connect: connection refused" 2024-06-01T02:01:37.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:37 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:01:37.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:37 smithi116 ceph-mon[29958]: Upgrade: Need to upgrade myself (mgr.y) 2024-06-01T02:01:37.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:37 smithi116 ceph-mon[29958]: Failing over to other MGR 2024-06-01T02:01:37.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:37 smithi116 ceph-mon[29958]: Standby manager daemon y started 2024-06-01T02:01:37.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:37 smithi116 ceph-mon[29958]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-06-01T02:01:37.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:37 smithi116 ceph-mon[29958]: mgrmap e21: x(active, starting, since 1.00193s), standbys: y 2024-06-01T02:01:37.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:37 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:01:37.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:37 smithi116 ceph-mon[25082]: Upgrade: Need to upgrade myself (mgr.y) 2024-06-01T02:01:37.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:37 smithi116 ceph-mon[25082]: Failing over to other MGR 2024-06-01T02:01:37.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:37 smithi116 ceph-mon[25082]: Standby manager daemon y started 2024-06-01T02:01:37.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:37 smithi116 ceph-mon[25082]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-06-01T02:01:37.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:37 smithi116 ceph-mon[25082]: mgrmap e21: x(active, starting, since 1.00193s), standbys: y 2024-06-01T02:01:37.774 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:37 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:01:37.774 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:37 smithi161 ceph-mon[26055]: Upgrade: Need to upgrade myself (mgr.y) 2024-06-01T02:01:37.774 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:37 smithi161 ceph-mon[26055]: Failing over to other MGR 2024-06-01T02:01:37.774 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:37 smithi161 ceph-mon[26055]: Standby manager daemon y started 2024-06-01T02:01:37.774 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:37 smithi161 ceph-mon[26055]: from='mgr.14427 172.21.15.116:0/2101155854' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-06-01T02:01:37.774 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:37 smithi161 ceph-mon[26055]: mgrmap e21: x(active, starting, since 1.00193s), standbys: y 2024-06-01T02:01:37.774 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:37 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:01:37.603+0000 7fc46c077200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-01T02:01:37.775 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:37 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:01:37.774+0000 7fc46c077200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-01T02:01:38.043 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:37 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T02:01:37.641+0000 7f300a609000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-01T02:01:38.044 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:37 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T02:01:37.723+0000 7f300a609000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-01T02:01:38.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:37 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:01:37.946+0000 7fc46c077200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-01T02:01:38.535 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:38 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:01:38.237+0000 7fc46c077200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-01T02:01:38.535 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:38 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:01:38.332+0000 7fc46c077200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-01T02:01:38.535 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:38 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:01:38.538+0000 7fc46c077200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-01T02:01:38.944 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:38 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:01:38.734+0000 7fc46c077200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-01T02:01:39.166 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:38 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T02:01:38.829+0000 7f300a609000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-01T02:01:39.166 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:38 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T02:01:38.991+0000 7f300a609000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-01T02:01:39.166 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:39 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T02:01:39.074+0000 7f300a609000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-01T02:01:39.695 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:39 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:01:39.338+0000 7fc46c077200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-01T02:01:39.793 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:39 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T02:01:39.438+0000 7f300a609000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-01T02:01:39.793 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:39 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T02:01:39.528+0000 7f300a609000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-01T02:01:39.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:39 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T02:01:39.622+0000 7f300a609000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-01T02:01:40.069 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:39 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:01:39.767+0000 7fc46c077200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-01T02:01:40.070 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:39 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:01:39.866+0000 7fc46c077200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-01T02:01:40.070 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:40 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:01:40.073+0000 7fc46c077200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-01T02:01:40.293 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:39 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T02:01:39.860+0000 7f300a609000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-01T02:01:40.779 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:40 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T02:01:40.299+0000 7f300a609000 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-01T02:01:40.944 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:40 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:01:40.606+0000 7fc46c077200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-01T02:01:40.944 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:40 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:01:40.700+0000 7fc46c077200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-01T02:01:41.043 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:40 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T02:01:40.781+0000 7f300a609000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-01T02:01:41.043 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:40 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T02:01:40.875+0000 7f300a609000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-01T02:01:41.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:40 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:01:40.947+0000 7fc46c077200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-01T02:01:41.195 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:41 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:01:41.088+0000 7fc46c077200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-01T02:01:41.543 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:41 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T02:01:41.173+0000 7f300a609000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-01T02:01:41.544 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:41 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T02:01:41.262+0000 7f300a609000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-01T02:01:41.621 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:41 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:01:41.201+0000 7fc46c077200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-01T02:01:41.621 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:41 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:01:41.319+0000 7fc46c077200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-01T02:01:41.621 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:41 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:01:41.625+0000 7fc46c077200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-01T02:01:41.805 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:41 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T02:01:41.676+0000 7f300a609000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-01T02:01:42.102 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:41 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T02:01:41.808+0000 7f300a609000 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-01T02:01:42.102 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:41 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T02:01:41.901+0000 7f300a609000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-01T02:01:42.102 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:42 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T02:01:42.016+0000 7f300a609000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-01T02:01:42.102 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:42 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T02:01:42.105+0000 7f300a609000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-01T02:01:42.519 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:42 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:01:42.237+0000 7fc46c077200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-01T02:01:42.519 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:42 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:01:42.457+0000 7fc46c077200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-01T02:01:42.519 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:42 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: [01/Jun/2024:02:01:42] ENGINE Bus STARTING 2024-06-01T02:01:42.793 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:42 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T02:01:42.671+0000 7f300a609000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-01T02:01:42.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:42 smithi116 ceph-mon[29958]: Active manager daemon x restarted 2024-06-01T02:01:42.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:42 smithi116 ceph-mon[29958]: Activating manager daemon x 2024-06-01T02:01:42.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:42 smithi116 ceph-mon[29958]: osdmap e97: 8 total, 8 up, 8 in 2024-06-01T02:01:42.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:42 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-01T02:01:42.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:42 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T02:01:42.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:42 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-01T02:01:42.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:42 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T02:01:42.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:42 smithi116 ceph-mon[25082]: Active manager daemon x restarted 2024-06-01T02:01:42.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:42 smithi116 ceph-mon[25082]: Activating manager daemon x 2024-06-01T02:01:42.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:42 smithi116 ceph-mon[25082]: osdmap e97: 8 total, 8 up, 8 in 2024-06-01T02:01:42.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:42 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-01T02:01:42.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:42 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T02:01:42.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:42 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-01T02:01:42.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:42 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T02:01:42.805 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:42 smithi161 ceph-mon[26055]: Active manager daemon x restarted 2024-06-01T02:01:42.805 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:42 smithi161 ceph-mon[26055]: Activating manager daemon x 2024-06-01T02:01:42.805 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:42 smithi161 ceph-mon[26055]: osdmap e97: 8 total, 8 up, 8 in 2024-06-01T02:01:42.805 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:42 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-01T02:01:42.805 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:42 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T02:01:42.805 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:42 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-01T02:01:42.805 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:42 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T02:01:42.806 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:42 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: CherryPy Checker: 2024-06-01T02:01:42.806 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:42 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: The Application mounted at '' has an empty config. 2024-06-01T02:01:42.806 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:42 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:01:42.806 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:42 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: [01/Jun/2024:02:01:42] ENGINE Serving on http://:::9283 2024-06-01T02:01:42.806 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:42 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: [01/Jun/2024:02:01:42] ENGINE Bus STARTED 2024-06-01T02:01:42.806 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:42 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: [01/Jun/2024:02:01:42] ENGINE Bus STOPPING 2024-06-01T02:01:42.806 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:42 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: [01/Jun/2024:02:01:42] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-06-01T02:01:42.806 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:42 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: [01/Jun/2024:02:01:42] ENGINE Bus STOPPED 2024-06-01T02:01:42.806 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:42 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: [01/Jun/2024:02:01:42] ENGINE Bus STARTING 2024-06-01T02:01:42.806 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:42 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: [01/Jun/2024:02:01:42] ENGINE Serving on http://:::9283 2024-06-01T02:01:42.806 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:42 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: [01/Jun/2024:02:01:42] ENGINE Bus STARTED 2024-06-01T02:01:43.149 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T02:01:43.044+0000 7f300a609000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-01T02:01:43.149 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: [01/Jun/2024:02:01:43] ENGINE Bus STARTING 2024-06-01T02:01:43.149 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: CherryPy Checker: 2024-06-01T02:01:43.149 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: The Application mounted at '' has an empty config. 2024-06-01T02:01:43.149 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: 2024-06-01T02:01:43.149 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: [01/Jun/2024:02:01:43] ENGINE Serving on http://:::9283 2024-06-01T02:01:43.149 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: [01/Jun/2024:02:01:43] ENGINE Bus STARTED 2024-06-01T02:01:43.793 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=error ts=2024-06-01T02:01:43.510Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=6 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.116:8443//api/prometheus_receiver\": dial tcp 172.21.15.116:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.161:8443/api/prometheus_receiver\": dial tcp 172.21.15.161:8443: connect: connection refused" 2024-06-01T02:01:43.794 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=warn ts=2024-06-01T02:01:43.512Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.116:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.116 because it doesn't contain any IP SANs" 2024-06-01T02:01:43.794 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=warn ts=2024-06-01T02:01:43.512Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.161:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.161 because it doesn't contain any IP SANs" 2024-06-01T02:01:43.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: mgrmap e22: x(active, starting, since 0.0605543s), standbys: y 2024-06-01T02:01:43.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T02:01:43.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T02:01:43.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T02:01:43.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-01T02:01:43.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-01T02:01:43.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T02:01:43.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T02:01:43.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T02:01:43.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T02:01:43.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T02:01:43.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T02:01:43.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T02:01:43.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T02:01:43.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-01T02:01:43.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-01T02:01:43.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-01T02:01:43.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: Manager daemon x is now available 2024-06-01T02:01:43.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:43.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:43.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:43.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:43.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi116", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:01:43.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:01:43.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi116", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:01:43.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:01:43.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:01:43.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:01:43.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: mgrmap e22: x(active, starting, since 0.0605543s), standbys: y 2024-06-01T02:01:43.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T02:01:43.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T02:01:43.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T02:01:43.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-01T02:01:43.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-01T02:01:43.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T02:01:43.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T02:01:43.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T02:01:43.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T02:01:43.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T02:01:43.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T02:01:43.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T02:01:43.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T02:01:43.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-01T02:01:43.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-01T02:01:43.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-01T02:01:43.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: Manager daemon x is now available 2024-06-01T02:01:43.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:43.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:43.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:43.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:43.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi116", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:01:43.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:01:43.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi116", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:01:43.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:01:43.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:01:43.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:01:43.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:01:43.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-01T02:01:43.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-01T02:01:43.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-01T02:01:43.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-01T02:01:43.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: Standby manager daemon y restarted 2024-06-01T02:01:43.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: Standby manager daemon y started 2024-06-01T02:01:43.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.116:0/1695060381' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-01T02:01:43.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.116:0/1695060381' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T02:01:43.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.116:0/1695060381' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-01T02:01:43.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.116:0/1695060381' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T02:01:43.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:43.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:43.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:01:43.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-01T02:01:43.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-01T02:01:43.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-01T02:01:43.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-01T02:01:43.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: Standby manager daemon y restarted 2024-06-01T02:01:43.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: Standby manager daemon y started 2024-06-01T02:01:43.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.116:0/1695060381' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-01T02:01:43.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.116:0/1695060381' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T02:01:43.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.116:0/1695060381' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-01T02:01:43.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.116:0/1695060381' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T02:01:43.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:43.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:43 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:43.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: mgrmap e22: x(active, starting, since 0.0605543s), standbys: y 2024-06-01T02:01:43.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T02:01:43.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T02:01:43.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T02:01:43.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-01T02:01:43.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-01T02:01:43.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T02:01:43.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T02:01:43.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T02:01:43.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T02:01:43.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T02:01:43.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T02:01:43.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T02:01:43.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T02:01:43.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-01T02:01:43.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-01T02:01:43.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-01T02:01:43.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: Manager daemon x is now available 2024-06-01T02:01:43.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:43.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:43.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:43.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:43.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi116", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:01:43.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:01:43.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi116", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:01:43.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:01:43.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:01:43.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:01:43.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:01:43.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-01T02:01:43.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-01T02:01:43.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-01T02:01:43.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-01T02:01:43.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: Standby manager daemon y restarted 2024-06-01T02:01:43.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: Standby manager daemon y started 2024-06-01T02:01:43.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.116:0/1695060381' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-01T02:01:43.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.116:0/1695060381' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T02:01:43.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.116:0/1695060381' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-01T02:01:43.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.116:0/1695060381' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T02:01:43.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:43.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:43 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:44.695 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:44 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:01:44] "GET /metrics HTTP/1.1" 200 35359 "" "Prometheus/2.33.4" 2024-06-01T02:01:44.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:44 smithi161 ceph-mon[26055]: Queued rgw.foo for migration 2024-06-01T02:01:44.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:44 smithi161 ceph-mon[26055]: Queued rgw.smpl for migration 2024-06-01T02:01:44.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:44 smithi161 ceph-mon[26055]: No Migration is needed for rgw spec: {'placement': {'count': 2}, 'service_id': 'foo', 'service_name': 'rgw.foo', 'service_type': 'rgw', 'spec': {'rgw_frontend_port': 8000, 'rgw_realm': 'r', 'rgw_zone': 'z'}} 2024-06-01T02:01:44.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:44 smithi161 ceph-mon[26055]: No Migration is needed for rgw spec: {'placement': {'count': 2}, 'service_id': 'smpl', 'service_name': 'rgw.smpl', 'service_type': 'rgw'} 2024-06-01T02:01:44.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:44 smithi161 ceph-mon[26055]: Updating smithi116:/etc/ceph/ceph.conf 2024-06-01T02:01:44.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:44 smithi161 ceph-mon[26055]: Updating smithi161:/etc/ceph/ceph.conf 2024-06-01T02:01:44.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:44 smithi161 ceph-mon[26055]: Updating smithi161:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-01T02:01:44.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:44 smithi161 ceph-mon[26055]: Updating smithi116:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-01T02:01:44.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:44 smithi161 ceph-mon[26055]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-06-01T02:01:44.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:44 smithi161 ceph-mon[26055]: Updating smithi116:/etc/ceph/ceph.client.admin.keyring 2024-06-01T02:01:44.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:44 smithi161 ceph-mon[26055]: mgrmap e23: x(active, since 1.07726s), standbys: y 2024-06-01T02:01:44.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:44 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:44.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:44 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:44.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:44 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:44.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:44 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:44.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:44 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:44.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:44 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:44.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:44 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:44.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:44 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:44.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:44 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:44.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:44 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:44.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:44 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:01:44.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:44 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:01:44.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[29958]: Queued rgw.foo for migration 2024-06-01T02:01:44.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[29958]: Queued rgw.smpl for migration 2024-06-01T02:01:44.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[29958]: No Migration is needed for rgw spec: {'placement': {'count': 2}, 'service_id': 'foo', 'service_name': 'rgw.foo', 'service_type': 'rgw', 'spec': {'rgw_frontend_port': 8000, 'rgw_realm': 'r', 'rgw_zone': 'z'}} 2024-06-01T02:01:44.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[29958]: No Migration is needed for rgw spec: {'placement': {'count': 2}, 'service_id': 'smpl', 'service_name': 'rgw.smpl', 'service_type': 'rgw'} 2024-06-01T02:01:44.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[29958]: Updating smithi116:/etc/ceph/ceph.conf 2024-06-01T02:01:44.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[29958]: Updating smithi161:/etc/ceph/ceph.conf 2024-06-01T02:01:44.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[29958]: Updating smithi161:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-01T02:01:44.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[29958]: Updating smithi116:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-01T02:01:44.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[29958]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-06-01T02:01:44.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[29958]: Updating smithi116:/etc/ceph/ceph.client.admin.keyring 2024-06-01T02:01:44.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[29958]: mgrmap e23: x(active, since 1.07726s), standbys: y 2024-06-01T02:01:44.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:44.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:44.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:44.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:44.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:44.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:44.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:44.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:44.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:44.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:44.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:01:44.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:01:44.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[25082]: Queued rgw.foo for migration 2024-06-01T02:01:44.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[25082]: Queued rgw.smpl for migration 2024-06-01T02:01:44.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[25082]: No Migration is needed for rgw spec: {'placement': {'count': 2}, 'service_id': 'foo', 'service_name': 'rgw.foo', 'service_type': 'rgw', 'spec': {'rgw_frontend_port': 8000, 'rgw_realm': 'r', 'rgw_zone': 'z'}} 2024-06-01T02:01:44.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[25082]: No Migration is needed for rgw spec: {'placement': {'count': 2}, 'service_id': 'smpl', 'service_name': 'rgw.smpl', 'service_type': 'rgw'} 2024-06-01T02:01:44.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[25082]: Updating smithi116:/etc/ceph/ceph.conf 2024-06-01T02:01:44.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[25082]: Updating smithi161:/etc/ceph/ceph.conf 2024-06-01T02:01:44.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[25082]: Updating smithi161:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-01T02:01:44.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[25082]: Updating smithi116:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-01T02:01:44.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[25082]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-06-01T02:01:44.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[25082]: Updating smithi116:/etc/ceph/ceph.client.admin.keyring 2024-06-01T02:01:44.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[25082]: mgrmap e23: x(active, since 1.07726s), standbys: y 2024-06-01T02:01:44.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:44.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:44.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:44.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:44.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:44.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:44.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:44.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:44.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:44.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:44.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:01:44.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:44 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:01:45.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:45 smithi116 ceph-mon[25082]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:01:45.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:45 smithi116 ceph-mon[25082]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:01:45.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:45 smithi116 ceph-mon[25082]: Updating smithi161:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-01T02:01:45.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:45 smithi116 ceph-mon[25082]: Updating smithi116:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-01T02:01:45.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:45 smithi116 ceph-mon[25082]: [01/Jun/2024:02:01:43] ENGINE Bus STARTING 2024-06-01T02:01:45.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:45 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:01:45.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:45 smithi116 ceph-mon[25082]: [01/Jun/2024:02:01:44] ENGINE Serving on https://172.21.15.161:7150 2024-06-01T02:01:45.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:45 smithi116 ceph-mon[25082]: [01/Jun/2024:02:01:44] ENGINE Serving on http://172.21.15.161:8765 2024-06-01T02:01:45.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:45 smithi116 ceph-mon[25082]: [01/Jun/2024:02:01:44] ENGINE Bus STARTED 2024-06-01T02:01:45.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:45 smithi116 ceph-mon[25082]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:01:45.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:45 smithi116 ceph-mon[25082]: Reconfiguring daemon agent.smithi116 on smithi116 2024-06-01T02:01:45.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:45 smithi116 ceph-mon[25082]: Deploying cephadm binary to smithi116 2024-06-01T02:01:45.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:45 smithi116 ceph-mon[25082]: mgrmap e24: x(active, since 2s), standbys: y 2024-06-01T02:01:45.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:45 smithi116 ceph-mon[29958]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:01:45.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:45 smithi116 ceph-mon[29958]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:01:45.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:45 smithi116 ceph-mon[29958]: Updating smithi161:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-01T02:01:45.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:45 smithi116 ceph-mon[29958]: Updating smithi116:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-01T02:01:45.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:45 smithi116 ceph-mon[29958]: [01/Jun/2024:02:01:43] ENGINE Bus STARTING 2024-06-01T02:01:45.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:45 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:01:45.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:45 smithi116 ceph-mon[29958]: [01/Jun/2024:02:01:44] ENGINE Serving on https://172.21.15.161:7150 2024-06-01T02:01:45.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:45 smithi116 ceph-mon[29958]: [01/Jun/2024:02:01:44] ENGINE Serving on http://172.21.15.161:8765 2024-06-01T02:01:45.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:45 smithi116 ceph-mon[29958]: [01/Jun/2024:02:01:44] ENGINE Bus STARTED 2024-06-01T02:01:45.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:45 smithi116 ceph-mon[29958]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:01:45.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:45 smithi116 ceph-mon[29958]: Reconfiguring daemon agent.smithi116 on smithi116 2024-06-01T02:01:45.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:45 smithi116 ceph-mon[29958]: Deploying cephadm binary to smithi116 2024-06-01T02:01:45.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:45 smithi116 ceph-mon[29958]: mgrmap e24: x(active, since 2s), standbys: y 2024-06-01T02:01:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:45 smithi161 ceph-mon[26055]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:01:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:45 smithi161 ceph-mon[26055]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:01:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:45 smithi161 ceph-mon[26055]: Updating smithi161:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-01T02:01:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:45 smithi161 ceph-mon[26055]: Updating smithi116:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-01T02:01:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:45 smithi161 ceph-mon[26055]: [01/Jun/2024:02:01:43] ENGINE Bus STARTING 2024-06-01T02:01:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:45 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:01:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:45 smithi161 ceph-mon[26055]: [01/Jun/2024:02:01:44] ENGINE Serving on https://172.21.15.161:7150 2024-06-01T02:01:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:45 smithi161 ceph-mon[26055]: [01/Jun/2024:02:01:44] ENGINE Serving on http://172.21.15.161:8765 2024-06-01T02:01:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:45 smithi161 ceph-mon[26055]: [01/Jun/2024:02:01:44] ENGINE Bus STARTED 2024-06-01T02:01:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:45 smithi161 ceph-mon[26055]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:01:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:45 smithi161 ceph-mon[26055]: Reconfiguring daemon agent.smithi116 on smithi116 2024-06-01T02:01:45.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:45 smithi161 ceph-mon[26055]: Deploying cephadm binary to smithi116 2024-06-01T02:01:45.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:45 smithi161 ceph-mon[26055]: mgrmap e24: x(active, since 2s), standbys: y 2024-06-01T02:01:47.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:47 smithi161 ceph-mon[26055]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:01:47.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:47 smithi161 ceph-mon[26055]: mgrmap e25: x(active, since 4s), standbys: y 2024-06-01T02:01:47.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:47 smithi116 ceph-mon[25082]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:01:47.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:47 smithi116 ceph-mon[25082]: mgrmap e25: x(active, since 4s), standbys: y 2024-06-01T02:01:47.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:47 smithi116 ceph-mon[29958]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:01:47.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:47 smithi116 ceph-mon[29958]: mgrmap e25: x(active, since 4s), standbys: y 2024-06-01T02:01:48.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:48 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:01:48] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-06-01T02:01:50.043 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:49 smithi116 systemd[1]: Stopping Ceph node-exporter.a for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:01:50.044 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:49 smithi116 bash[63223]: Error: no container with name or ID "ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter.a" found: no such container 2024-06-01T02:01:50.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:50 smithi161 ceph-mon[26055]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:01:50.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:50 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:50.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:50 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:50.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:50 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:50.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:50 smithi161 ceph-mon[26055]: Reconfiguring node-exporter.a (dependencies changed)... 2024-06-01T02:01:50.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:50 smithi161 ceph-mon[26055]: Reconfiguring daemon node-exporter.a on smithi116 2024-06-01T02:01:50.528 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:50 smithi116 ceph-mon[29958]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:01:50.528 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:50 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:50.528 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:50 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:50.528 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:50 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:50.528 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:50 smithi116 ceph-mon[29958]: Reconfiguring node-exporter.a (dependencies changed)... 2024-06-01T02:01:50.528 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:50 smithi116 ceph-mon[29958]: Reconfiguring daemon node-exporter.a on smithi116 2024-06-01T02:01:50.528 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:50 smithi116 ceph-mon[25082]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:01:50.529 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:50 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:50.529 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:50 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:50.529 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:50 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:50.529 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:50 smithi116 ceph-mon[25082]: Reconfiguring node-exporter.a (dependencies changed)... 2024-06-01T02:01:50.529 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:50 smithi116 ceph-mon[25082]: Reconfiguring daemon node-exporter.a on smithi116 2024-06-01T02:01:50.793 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:50 smithi116 podman[63237]: 2024-06-01 02:01:50.759221585 +0000 UTC m=+0.991448688 container died 35a72a0577d412362116fcfa118e4b15a15303e0d5d2152a2ede4ea54e3022a4 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-06-01T02:01:51.377 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:51 smithi116 podman[63237]: 2024-06-01 02:01:51.130488807 +0000 UTC m=+1.362715903 container cleanup 35a72a0577d412362116fcfa118e4b15a15303e0d5d2152a2ede4ea54e3022a4 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-06-01T02:01:51.378 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:51 smithi116 bash[63237]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a 2024-06-01T02:01:51.378 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:51 smithi116 bash[63290]: Error: no container with name or ID "ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter.a" found: no such container 2024-06-01T02:01:51.663 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:51 smithi116 podman[63262]: 2024-06-01 02:01:51.381915377 +0000 UTC m=+1.155583978 container remove 35a72a0577d412362116fcfa118e4b15a15303e0d5d2152a2ede4ea54e3022a4 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-06-01T02:01:51.663 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:51 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-06-01T02:01:51.663 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:51 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-06-01T02:01:51.664 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:51 smithi116 systemd[1]: Stopped Ceph node-exporter.a for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:01:51.664 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:51 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@node-exporter.a.service: Consumed 2.689s CPU time. 2024-06-01T02:01:51.664 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:51 smithi116 systemd[1]: Starting Ceph node-exporter.a for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:01:52.414 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-mon[25082]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-06-01T02:01:52.414 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-mon[29958]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-06-01T02:01:52.414 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 podman[63405]: 2024-06-01 02:01:52.179864691 +0000 UTC m=+0.018404429 image pull 1dbe0e931976487e20e5cfb272087e08a9779c88fd5e9617ed7042dd9751ec26 quay.io/prometheus/node-exporter:v1.3.1 2024-06-01T02:01:52.414 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 podman[63405]: 2024-06-01 02:01:52.295741716 +0000 UTC m=+0.134281442 container create e347748d17bb33ab1f10ea6444ac5c3f24a297ee40e4451df4116a2bdd1c8603 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-06-01T02:01:52.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:52 smithi161 ceph-mon[26055]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-06-01T02:01:52.831 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 podman[63405]: 2024-06-01 02:01:52.556706194 +0000 UTC m=+0.395245919 container init e347748d17bb33ab1f10ea6444ac5c3f24a297ee40e4451df4116a2bdd1c8603 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-06-01T02:01:52.832 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 podman[63405]: 2024-06-01 02:01:52.559645093 +0000 UTC m=+0.398184813 container start e347748d17bb33ab1f10ea6444ac5c3f24a297ee40e4451df4116a2bdd1c8603 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-06-01T02:01:52.832 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.562Z caller=node_exporter.go:182 level=info msg="Starting node_exporter" version="(version=1.3.1, branch=HEAD, revision=a2321e7b940ddcff26873612bccdf7cd4c42b6b6)" 2024-06-01T02:01:52.832 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.562Z caller=node_exporter.go:183 level=info msg="Build context" build_context="(go=go1.17.3, user=root@243aafa5525c, date=20211205-11:09:49)" 2024-06-01T02:01:52.832 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.563Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+)($|/) 2024-06-01T02:01:52.832 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.563Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-06-01T02:01:52.832 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.563Z caller=node_exporter.go:108 level=info msg="Enabled collectors" 2024-06-01T02:01:52.832 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.563Z caller=node_exporter.go:115 level=info collector=arp 2024-06-01T02:01:52.832 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.563Z caller=node_exporter.go:115 level=info collector=bcache 2024-06-01T02:01:52.832 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.563Z caller=node_exporter.go:115 level=info collector=bonding 2024-06-01T02:01:52.832 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.563Z caller=node_exporter.go:115 level=info collector=btrfs 2024-06-01T02:01:52.832 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.563Z caller=node_exporter.go:115 level=info collector=conntrack 2024-06-01T02:01:52.832 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.563Z caller=node_exporter.go:115 level=info collector=cpu 2024-06-01T02:01:52.833 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.563Z caller=node_exporter.go:115 level=info collector=cpufreq 2024-06-01T02:01:52.833 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.563Z caller=node_exporter.go:115 level=info collector=diskstats 2024-06-01T02:01:52.833 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.563Z caller=node_exporter.go:115 level=info collector=dmi 2024-06-01T02:01:52.833 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.563Z caller=node_exporter.go:115 level=info collector=edac 2024-06-01T02:01:52.833 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.563Z caller=node_exporter.go:115 level=info collector=entropy 2024-06-01T02:01:52.834 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.563Z caller=node_exporter.go:115 level=info collector=fibrechannel 2024-06-01T02:01:52.834 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.563Z caller=node_exporter.go:115 level=info collector=filefd 2024-06-01T02:01:52.834 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.563Z caller=node_exporter.go:115 level=info collector=filesystem 2024-06-01T02:01:52.834 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.563Z caller=node_exporter.go:115 level=info collector=hwmon 2024-06-01T02:01:52.834 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.563Z caller=node_exporter.go:115 level=info collector=infiniband 2024-06-01T02:01:52.834 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.563Z caller=node_exporter.go:115 level=info collector=ipvs 2024-06-01T02:01:52.834 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.563Z caller=node_exporter.go:115 level=info collector=loadavg 2024-06-01T02:01:52.834 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.563Z caller=node_exporter.go:115 level=info collector=mdadm 2024-06-01T02:01:52.834 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.563Z caller=node_exporter.go:115 level=info collector=meminfo 2024-06-01T02:01:52.834 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.563Z caller=node_exporter.go:115 level=info collector=netclass 2024-06-01T02:01:52.834 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.563Z caller=node_exporter.go:115 level=info collector=netdev 2024-06-01T02:01:52.835 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.563Z caller=node_exporter.go:115 level=info collector=netstat 2024-06-01T02:01:52.835 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.563Z caller=node_exporter.go:115 level=info collector=nfs 2024-06-01T02:01:52.835 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.563Z caller=node_exporter.go:115 level=info collector=nfsd 2024-06-01T02:01:52.835 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.563Z caller=node_exporter.go:115 level=info collector=nvme 2024-06-01T02:01:52.835 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.564Z caller=node_exporter.go:115 level=info collector=os 2024-06-01T02:01:52.835 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.564Z caller=node_exporter.go:115 level=info collector=powersupplyclass 2024-06-01T02:01:52.835 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.564Z caller=node_exporter.go:115 level=info collector=pressure 2024-06-01T02:01:52.835 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.564Z caller=node_exporter.go:115 level=info collector=rapl 2024-06-01T02:01:52.835 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.564Z caller=node_exporter.go:115 level=info collector=schedstat 2024-06-01T02:01:52.835 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.564Z caller=node_exporter.go:115 level=info collector=sockstat 2024-06-01T02:01:52.835 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.564Z caller=node_exporter.go:115 level=info collector=softnet 2024-06-01T02:01:52.835 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.564Z caller=node_exporter.go:115 level=info collector=stat 2024-06-01T02:01:52.835 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.564Z caller=node_exporter.go:115 level=info collector=tapestats 2024-06-01T02:01:52.836 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.564Z caller=node_exporter.go:115 level=info collector=textfile 2024-06-01T02:01:52.836 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.564Z caller=node_exporter.go:115 level=info collector=thermal_zone 2024-06-01T02:01:52.836 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.564Z caller=node_exporter.go:115 level=info collector=time 2024-06-01T02:01:52.836 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.564Z caller=node_exporter.go:115 level=info collector=udp_queues 2024-06-01T02:01:52.836 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.564Z caller=node_exporter.go:115 level=info collector=uname 2024-06-01T02:01:52.836 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.564Z caller=node_exporter.go:115 level=info collector=vmstat 2024-06-01T02:01:52.836 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.564Z caller=node_exporter.go:115 level=info collector=xfs 2024-06-01T02:01:52.836 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.564Z caller=node_exporter.go:115 level=info collector=zfs 2024-06-01T02:01:52.836 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.564Z caller=node_exporter.go:199 level=info msg="Listening on" address=:9100 2024-06-01T02:01:52.836 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a[63438]: ts=2024-06-01T02:01:52.564Z caller=tls_config.go:195 level=info msg="TLS is disabled." http2=false 2024-06-01T02:01:52.836 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 bash[63405]: e347748d17bb33ab1f10ea6444ac5c3f24a297ee40e4451df4116a2bdd1c8603 2024-06-01T02:01:52.836 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:01:52 smithi116 systemd[1]: Started Ceph node-exporter.a for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:01:53.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:53 smithi116 ceph-mon[25082]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-06-01T02:01:53.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:53 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:53.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:53 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:53.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:53 smithi116 ceph-mon[25082]: Reconfiguring alertmanager.a (dependencies changed)... 2024-06-01T02:01:53.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:53 smithi116 ceph-mon[25082]: Reconfiguring daemon alertmanager.a on smithi116 2024-06-01T02:01:53.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:53 smithi116 ceph-mon[29958]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-06-01T02:01:53.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:53 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:53.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:53 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:53.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:53 smithi116 ceph-mon[29958]: Reconfiguring alertmanager.a (dependencies changed)... 2024-06-01T02:01:53.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:53 smithi116 ceph-mon[29958]: Reconfiguring daemon alertmanager.a on smithi116 2024-06-01T02:01:53.795 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:01:53 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=error ts=2024-06-01T02:01:53.510Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=6 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.116:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.116 because it doesn't contain any IP SANs; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://172.21.15.161:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.161 because it doesn't contain any IP SANs" 2024-06-01T02:01:53.795 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:01:53 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=warn ts=2024-06-01T02:01:53.512Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.116:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.116 because it doesn't contain any IP SANs" 2024-06-01T02:01:53.795 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:01:53 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=warn ts=2024-06-01T02:01:53.512Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.161:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.161 because it doesn't contain any IP SANs" 2024-06-01T02:01:53.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:53 smithi161 ceph-mon[26055]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-06-01T02:01:53.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:53 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:53.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:53 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:01:53.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:53 smithi161 ceph-mon[26055]: Reconfiguring alertmanager.a (dependencies changed)... 2024-06-01T02:01:53.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:53 smithi161 ceph-mon[26055]: Reconfiguring daemon alertmanager.a on smithi116 2024-06-01T02:01:54.695 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:01:54 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:01:54] "GET /metrics HTTP/1.1" 200 35359 "" "Prometheus/2.33.4" 2024-06-01T02:01:55.510 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:55 smithi116 ceph-mon[25082]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:01:55.510 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:55 smithi116 ceph-mon[29958]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:01:55.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:55 smithi161 ceph-mon[26055]: from='client.24728 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:01:56.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:56 smithi116 ceph-mon[25082]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-06-01T02:01:56.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:56 smithi116 ceph-mon[29958]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-06-01T02:01:56.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:56 smithi161 ceph-mon[26055]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-06-01T02:01:57.524 INFO:teuthology.orchestra.run.smithi116.stdout:true 2024-06-01T02:01:58.195 INFO:teuthology.orchestra.run.smithi116.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-01T02:01:58.195 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi116 smithi116 starting - - - - 2024-06-01T02:01:58.195 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi161 smithi161 running 22s ago 10m - - 2024-06-01T02:01:58.195 INFO:teuthology.orchestra.run.smithi116.stdout:alertmanager.a smithi116 *:9093,9094 running (4m) 35s ago 5m 26.5M - ba2b418f427c 7cd3733dd584 2024-06-01T02:01:58.196 INFO:teuthology.orchestra.run.smithi116.stdout:grafana.a smithi161 *:3000 running (4m) 22s ago 4m 52.9M - 8.3.5 dad864ee21e9 f4be12ac83e0 2024-06-01T02:01:58.196 INFO:teuthology.orchestra.run.smithi116.stdout:iscsi.foo.smithi116.gjuxzb smithi116 running (3m) 35s ago 3m 48.5M - 3.5 e1d6a67b021e 5b3b199ca2db 2024-06-01T02:01:58.196 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.x smithi161 *:8443,9283 starting - - - - 2024-06-01T02:01:58.196 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.y smithi116 *:9283 running (11m) 35s ago 11m 463M - 17.2.0 e1d6a67b021e 3c1beb78ef72 2024-06-01T02:01:58.196 INFO:teuthology.orchestra.run.smithi116.stdout:mon.a smithi116 running (11m) 35s ago 11m 51.7M 2048M 17.2.0 e1d6a67b021e c821092a538a 2024-06-01T02:01:58.196 INFO:teuthology.orchestra.run.smithi116.stdout:mon.b smithi161 running (10m) 22s ago 10m 38.9M 2048M 17.2.0 e1d6a67b021e 9011986aa6e3 2024-06-01T02:01:58.196 INFO:teuthology.orchestra.run.smithi116.stdout:mon.c smithi116 running (10m) 35s ago 10m 44.1M 2048M 17.2.0 e1d6a67b021e 4a6fe0b16d63 2024-06-01T02:01:58.196 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.a smithi116 *:9100 starting - - - - 2024-06-01T02:01:58.196 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.b smithi161 *:9100 running (5m) 22s ago 5m 17.0M - 1dbe0e931976 4456eaae965b 2024-06-01T02:01:58.196 INFO:teuthology.orchestra.run.smithi116.stdout:osd.0 smithi116 running (9m) 35s ago 9m 53.1M 4096M 17.2.0 e1d6a67b021e 0cc929b0ecad 2024-06-01T02:01:58.196 INFO:teuthology.orchestra.run.smithi116.stdout:osd.1 smithi116 running (9m) 35s ago 9m 54.2M 4096M 17.2.0 e1d6a67b021e 6ff7fc5abe58 2024-06-01T02:01:58.196 INFO:teuthology.orchestra.run.smithi116.stdout:osd.2 smithi116 running (8m) 35s ago 8m 49.9M 4096M 17.2.0 e1d6a67b021e 6aecbb10256b 2024-06-01T02:01:58.196 INFO:teuthology.orchestra.run.smithi116.stdout:osd.3 smithi116 running (8m) 35s ago 8m 52.9M 4096M 17.2.0 e1d6a67b021e a802290f5290 2024-06-01T02:01:58.197 INFO:teuthology.orchestra.run.smithi116.stdout:osd.4 smithi161 running (7m) 22s ago 7m 55.2M 4096M 17.2.0 e1d6a67b021e 70b3f89b0bd9 2024-06-01T02:01:58.197 INFO:teuthology.orchestra.run.smithi116.stdout:osd.5 smithi161 running (7m) 22s ago 7m 49.6M 4096M 17.2.0 e1d6a67b021e 68358b78a25d 2024-06-01T02:01:58.197 INFO:teuthology.orchestra.run.smithi116.stdout:osd.6 smithi161 running (7m) 22s ago 7m 48.4M 4096M 17.2.0 e1d6a67b021e 50abeb0551e7 2024-06-01T02:01:58.197 INFO:teuthology.orchestra.run.smithi116.stdout:osd.7 smithi161 running (6m) 22s ago 6m 52.7M 4096M 17.2.0 e1d6a67b021e 6004e6983735 2024-06-01T02:01:58.197 INFO:teuthology.orchestra.run.smithi116.stdout:prometheus.a smithi161 *:9095 running (4m) 22s ago 5m 66.6M - 514e6a882f6e bea6bf7e7083 2024-06-01T02:01:58.197 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi116.vzjrxj smithi116 *:8000 running (4m) 35s ago 4m 72.5M - 17.2.0 e1d6a67b021e df4fe3c1dbc3 2024-06-01T02:01:58.197 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi161.ashqxz smithi161 *:8000 running (4m) 22s ago 4m 73.7M - 17.2.0 e1d6a67b021e 3eecc1f3e2d3 2024-06-01T02:01:58.197 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi116.hokbmf smithi116 *:80 running (3m) 35s ago 3m 73.4M - 17.2.0 e1d6a67b021e 10950ed75dbd 2024-06-01T02:01:58.197 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi161.axjlzw smithi161 *:80 running (3m) 22s ago 3m 72.5M - 17.2.0 e1d6a67b021e 0b5d5ce47772 2024-06-01T02:01:58.525 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:58 smithi116 ceph-mon[25082]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-01T02:01:58.526 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:58 smithi116 ceph-mon[25082]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:01:58.526 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:58 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:01:58.526 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:58 smithi116 ceph-mon[29958]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-01T02:01:58.526 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:58 smithi116 ceph-mon[29958]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:01:58.526 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:58 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:01:58.592 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:01:58.592 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": { 2024-06-01T02:01:58.593 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 3 2024-06-01T02:01:58.593 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:01:58.593 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": { 2024-06-01T02:01:58.593 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 1, 2024-06-01T02:01:58.593 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 1 2024-06-01T02:01:58.593 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:01:58.593 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": { 2024-06-01T02:01:58.593 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2024-06-01T02:01:58.593 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:01:58.593 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": {}, 2024-06-01T02:01:58.593 INFO:teuthology.orchestra.run.smithi116.stdout: "rgw": { 2024-06-01T02:01:58.593 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-06-01T02:01:58.593 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:01:58.593 INFO:teuthology.orchestra.run.smithi116.stdout: "overall": { 2024-06-01T02:01:58.593 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 16, 2024-06-01T02:01:58.593 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 1 2024-06-01T02:01:58.594 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T02:01:58.594 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:01:58.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:58 smithi161 ceph-mon[26055]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-01T02:01:58.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:58 smithi161 ceph-mon[26055]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:01:58.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:58 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:01:58.786 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:01:58 smithi116 systemd[1]: Stopping Ceph alertmanager.a for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:01:58.787 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:01:58 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:01:58] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-06-01T02:01:58.958 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:01:58.958 INFO:teuthology.orchestra.run.smithi116.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-01T02:01:58.958 INFO:teuthology.orchestra.run.smithi116.stdout: "in_progress": true, 2024-06-01T02:01:58.958 INFO:teuthology.orchestra.run.smithi116.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-01T02:01:58.959 INFO:teuthology.orchestra.run.smithi116.stdout: "services_complete": [], 2024-06-01T02:01:58.959 INFO:teuthology.orchestra.run.smithi116.stdout: "progress": "", 2024-06-01T02:01:58.959 INFO:teuthology.orchestra.run.smithi116.stdout: "message": "", 2024-06-01T02:01:58.959 INFO:teuthology.orchestra.run.smithi116.stdout: "is_paused": false 2024-06-01T02:01:58.959 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:01:59.043 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:01:58 smithi116 bash[64088]: Error: no container with name or ID "ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager.a" found: no such container 2024-06-01T02:01:59.044 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:01:58 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[50997]: level=info ts=2024-06-01T02:01:58.903Z caller=main.go:557 msg="Received SIGTERM, exiting gracefully..." 2024-06-01T02:01:59.044 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:01:58 smithi116 podman[64108]: 2024-06-01 02:01:58.924170038 +0000 UTC m=+0.124274906 container died 7cd3733dd5843e182b46675bed8d086b6272114cc448b27add914fdd6dddafaf (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-01T02:01:59.351 INFO:teuthology.orchestra.run.smithi116.stdout:HEALTH_OK 2024-06-01T02:01:59.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:59 smithi161 ceph-mon[26055]: from='client.25117 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:01:59.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:59 smithi161 ceph-mon[26055]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:01:59.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:01:59 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/3229737501' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:01:59.726 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:59 smithi116 ceph-mon[29958]: from='client.25117 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:01:59.726 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:59 smithi116 ceph-mon[29958]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:01:59.727 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:01:59 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/3229737501' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:01:59.727 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:59 smithi116 ceph-mon[25082]: from='client.25117 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:01:59.727 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:59 smithi116 ceph-mon[25082]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:01:59.727 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:01:59 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/3229737501' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:01:59.727 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:01:59 smithi116 podman[64108]: 2024-06-01 02:01:59.54617146 +0000 UTC m=+0.746276317 container cleanup 7cd3733dd5843e182b46675bed8d086b6272114cc448b27add914fdd6dddafaf (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-01T02:01:59.727 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:01:59 smithi116 bash[64108]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a 2024-06-01T02:02:00.043 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:01:59 smithi116 podman[64132]: 2024-06-01 02:01:59.731093217 +0000 UTC m=+0.806401832 container remove 7cd3733dd5843e182b46675bed8d086b6272114cc448b27add914fdd6dddafaf (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-01T02:02:00.043 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:01:59 smithi116 podman[64132]: 2024-06-01 02:01:59.831758545 +0000 UTC m=+0.907067098 volume remove be2bad1a8e29ad772204f5a766e30a6ba23e6d554dc18e535265dc2fd20be991 2024-06-01T02:02:00.044 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:01:59 smithi116 bash[64172]: Error: no container with name or ID "ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager.a" found: no such container 2024-06-01T02:02:00.044 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:01:59 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-06-01T02:02:00.044 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:01:59 smithi116 systemd[1]: Stopped Ceph alertmanager.a for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:02:00.044 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:01:59 smithi116 systemd[1]: Starting Ceph alertmanager.a for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:02:00.481 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:00 smithi116 ceph-mon[29958]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-01T02:02:00.481 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:00 smithi116 ceph-mon[29958]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:02:00.481 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:00 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1183528187' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:02:00.481 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:00 smithi116 ceph-mon[25082]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-01T02:02:00.481 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:00 smithi116 ceph-mon[25082]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:02:00.482 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:00 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1183528187' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:02:00.482 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:02:00 smithi116 podman[64275]: 2024-06-01 02:02:00.393194115 +0000 UTC m=+0.114175771 volume create ce7f7ac35efe9a0fb1e8ae759931fddc0c167ea28ca054f7e9da477ae1300ce2 2024-06-01T02:02:00.482 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:02:00 smithi116 podman[64275]: 2024-06-01 02:02:00.296379728 +0000 UTC m=+0.017361343 image pull ba2b418f427c0636d654de8757e830c80168e76482bcc46bb2138e569d6c91d4 quay.io/prometheus/alertmanager:v0.23.0 2024-06-01T02:02:00.648 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:00 smithi161 ceph-mon[26055]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-01T02:02:00.648 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:00 smithi161 ceph-mon[26055]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:02:00.648 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:00 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1183528187' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:02:00.793 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:02:00 smithi116 podman[64275]: 2024-06-01 02:02:00.485285008 +0000 UTC m=+0.206266664 container create e0c297162dede4f605e15650dd5c8aee31163d9400299110e5d10a9ce1997313 (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-01T02:02:01.247 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:02:00 smithi116 podman[64275]: 2024-06-01 02:02:00.813359619 +0000 UTC m=+0.534341233 container init e0c297162dede4f605e15650dd5c8aee31163d9400299110e5d10a9ce1997313 (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-01T02:02:01.248 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:02:00 smithi116 podman[64275]: 2024-06-01 02:02:00.81613515 +0000 UTC m=+0.537116764 container start e0c297162dede4f605e15650dd5c8aee31163d9400299110e5d10a9ce1997313 (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-01T02:02:01.248 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:02:00 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[64300]: level=info ts=2024-06-01T02:02:00.826Z caller=main.go:225 msg="Starting Alertmanager" version="(version=0.23.0, branch=HEAD, revision=61046b17771a57cfd4c4a51be370ab930a4d7d54)" 2024-06-01T02:02:01.248 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:02:00 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[64300]: level=info ts=2024-06-01T02:02:00.826Z caller=main.go:226 build_context="(go=go1.16.7, user=root@e21a959be8d2, date=20210825-10:48:55)" 2024-06-01T02:02:01.248 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:02:00 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[64300]: level=info ts=2024-06-01T02:02:00.828Z caller=cluster.go:184 component=cluster msg="setting advertise address explicitly" addr=172.21.15.116 port=9094 2024-06-01T02:02:01.248 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:02:00 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[64300]: level=info ts=2024-06-01T02:02:00.828Z caller=cluster.go:671 component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-06-01T02:02:01.248 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:02:00 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[64300]: level=info ts=2024-06-01T02:02:00.853Z caller=coordinator.go:113 component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-06-01T02:02:01.248 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:02:00 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[64300]: level=info ts=2024-06-01T02:02:00.854Z caller=coordinator.go:126 component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-06-01T02:02:01.248 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:02:00 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[64300]: level=info ts=2024-06-01T02:02:00.856Z caller=main.go:518 msg=Listening address=:9093 2024-06-01T02:02:01.248 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:02:00 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[64300]: level=info ts=2024-06-01T02:02:00.856Z caller=tls_config.go:191 msg="TLS is disabled." http2=false 2024-06-01T02:02:01.248 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:02:00 smithi116 bash[64275]: e0c297162dede4f605e15650dd5c8aee31163d9400299110e5d10a9ce1997313 2024-06-01T02:02:01.248 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:02:00 smithi116 systemd[1]: Started Ceph alertmanager.a for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:02:02.194 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:01 smithi116 ceph-mon[25082]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-01T02:02:02.194 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:01 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:02.194 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:01 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:02.194 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:01 smithi116 ceph-mon[25082]: Reconfiguring iscsi.foo.smithi116.gjuxzb (dependencies changed)... 2024-06-01T02:02:02.194 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:01 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi116.gjuxzb", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-01T02:02:02.194 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:01 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi116.gjuxzb", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-01T02:02:02.194 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:01 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:02:02.195 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:01 smithi116 ceph-mon[25082]: Reconfiguring daemon iscsi.foo.smithi116.gjuxzb on smithi116 2024-06-01T02:02:02.195 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:01 smithi116 ceph-mon[29958]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-01T02:02:02.195 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:01 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:02.195 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:01 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:02.195 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:01 smithi116 ceph-mon[29958]: Reconfiguring iscsi.foo.smithi116.gjuxzb (dependencies changed)... 2024-06-01T02:02:02.195 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:01 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi116.gjuxzb", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-01T02:02:02.195 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:01 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi116.gjuxzb", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-01T02:02:02.195 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:01 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:02:02.195 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:01 smithi116 ceph-mon[29958]: Reconfiguring daemon iscsi.foo.smithi116.gjuxzb on smithi116 2024-06-01T02:02:02.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:01 smithi161 ceph-mon[26055]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-01T02:02:02.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:01 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:02.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:01 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:02.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:01 smithi161 ceph-mon[26055]: Reconfiguring iscsi.foo.smithi116.gjuxzb (dependencies changed)... 2024-06-01T02:02:02.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:01 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi116.gjuxzb", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-01T02:02:02.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:01 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi116.gjuxzb", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-01T02:02:02.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:01 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:02:02.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:01 smithi161 ceph-mon[26055]: Reconfiguring daemon iscsi.foo.smithi116.gjuxzb on smithi116 2024-06-01T02:02:02.695 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:02 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T02:02:02.315Z caller=manager.go:609 level=warn component="rule manager" group=pools msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on(pool_id) group_right()\n ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: |\n Pool '{{ $labels.name }}' will be full in less than 5 days assuming the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed it's capacity\n" err="found duplicate series for the match group {pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"172.21.15.161:9283\", job=\"ceph\", pool_id=\"1\"}, {instance=\"172.21.15.116:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:02:03.043 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:02:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[64300]: level=info ts=2024-06-01T02:02:02.829Z caller=cluster.go:696 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000097737s 2024-06-01T02:02:04.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:03 smithi161 ceph-mon[26055]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:02:04.286 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:03 smithi116 ceph-mon[29958]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:02:04.286 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:03 smithi116 ceph-mon[25082]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:02:04.594 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:02:04] "GET /metrics HTTP/1.1" 200 38160 "" "Prometheus/2.33.4" 2024-06-01T02:02:04.945 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:02:04.672+0000 7fc3f1bfd700 -1 log_channel(cephadm) log [ERR] : cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi116.gjuxzb ... 2024-06-01T02:02:04.945 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: Creating ceph-iscsi config... 2024-06-01T02:02:04.945 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/iscsi-gateway.cfg 2024-06-01T02:02:04.945 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/tcmu-runner-entrypoint.sh 2024-06-01T02:02:04.946 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: Non-zero exit code 1 from systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb 2024-06-01T02:02:04.946 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: systemctl: stderr Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:02:04.946 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: systemctl: stderr See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:02:04.946 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: Traceback (most recent call last): 2024-06-01T02:02:04.946 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-06-01T02:02:04.946 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: return _run_code(code, main_globals, None, 2024-06-01T02:02:04.947 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-06-01T02:02:04.947 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: exec(code, run_globals) 2024-06-01T02:02:04.947 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5618, in 2024-06-01T02:02:04.947 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5606, in main 2024-06-01T02:02:04.947 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3096, in command_deploy_from 2024-06-01T02:02:04.947 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3131, in _common_deploy 2024-06-01T02:02:04.948 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3151, in _deploy_daemon_container 2024-06-01T02:02:04.948 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 1123, in deploy_daemon 2024-06-01T02:02:04.948 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-06-01T02:02:04.948 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: RuntimeError: Failed command: systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb: Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:02:04.949 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:02:04.949 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: Traceback (most recent call last): 2024-06-01T02:02:04.949 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1114, in _check_daemons 2024-06-01T02:02:04.949 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: self.mgr._daemon_action(daemon_spec, action=action) 2024-06-01T02:02:04.949 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: File "/usr/share/ceph/mgr/cephadm/module.py", line 2457, in _daemon_action 2024-06-01T02:02:04.949 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: CephadmServe(self)._create_daemon(daemon_spec, reconfig=(action == 'reconfig'))) 2024-06-01T02:02:04.949 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: File "/usr/share/ceph/mgr/cephadm/module.py", line 779, in wait_async 2024-06-01T02:02:04.949 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: return self.event_loop.get_result(coro, timeout) 2024-06-01T02:02:04.949 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: File "/usr/share/ceph/mgr/cephadm/ssh.py", line 136, in get_result 2024-06-01T02:02:04.949 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: return future.result(timeout) 2024-06-01T02:02:04.949 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: File "/lib64/python3.6/concurrent/futures/_base.py", line 432, in result 2024-06-01T02:02:04.949 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: return self.__get_result() 2024-06-01T02:02:04.949 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: File "/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result 2024-06-01T02:02:04.950 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: raise self._exception 2024-06-01T02:02:04.950 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1377, in _create_daemon 2024-06-01T02:02:04.950 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: use_current_daemon_image=reconfig, 2024-06-01T02:02:04.950 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1692, in _run_cephadm 2024-06-01T02:02:04.950 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: f'cephadm exited with an error code: {code}, stderr: {err}') 2024-06-01T02:02:04.950 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi116.gjuxzb ... 2024-06-01T02:02:04.950 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: Creating ceph-iscsi config... 2024-06-01T02:02:04.950 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/iscsi-gateway.cfg 2024-06-01T02:02:04.950 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/tcmu-runner-entrypoint.sh 2024-06-01T02:02:04.950 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: Non-zero exit code 1 from systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb 2024-06-01T02:02:04.950 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: systemctl: stderr Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:02:04.951 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: systemctl: stderr See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:02:04.951 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: Traceback (most recent call last): 2024-06-01T02:02:04.951 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-06-01T02:02:04.951 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: return _run_code(code, main_globals, None, 2024-06-01T02:02:04.951 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-06-01T02:02:04.951 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: exec(code, run_globals) 2024-06-01T02:02:04.951 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5618, in 2024-06-01T02:02:04.951 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5606, in main 2024-06-01T02:02:04.951 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3096, in command_deploy_from 2024-06-01T02:02:04.951 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3131, in _common_deploy 2024-06-01T02:02:04.951 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3151, in _deploy_daemon_container 2024-06-01T02:02:04.951 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 1123, in deploy_daemon 2024-06-01T02:02:04.951 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-06-01T02:02:04.952 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: RuntimeError: Failed command: systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb: Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:02:04.952 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:02:05.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:05 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:02:05.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:05 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:02:05.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:05 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:02:05.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:05 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:02:05.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:05 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:02:05.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:05 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:02:06.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:06 smithi161 ceph-mon[26055]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:02:06.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:06 smithi161 ceph-mon[26055]: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi116.gjuxzb ... 2024-06-01T02:02:06.446 INFO:journalctl@ceph.mon.b.smithi161.stdout: Creating ceph-iscsi config... 2024-06-01T02:02:06.446 INFO:journalctl@ceph.mon.b.smithi161.stdout: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/iscsi-gateway.cfg 2024-06-01T02:02:06.446 INFO:journalctl@ceph.mon.b.smithi161.stdout: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/tcmu-runner-entrypoint.sh 2024-06-01T02:02:06.446 INFO:journalctl@ceph.mon.b.smithi161.stdout: Non-zero exit code 1 from systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb 2024-06-01T02:02:06.446 INFO:journalctl@ceph.mon.b.smithi161.stdout: systemctl: stderr Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:02:06.447 INFO:journalctl@ceph.mon.b.smithi161.stdout: systemctl: stderr See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:02:06.450 INFO:journalctl@ceph.mon.b.smithi161.stdout: Traceback (most recent call last): 2024-06-01T02:02:06.450 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-06-01T02:02:06.450 INFO:journalctl@ceph.mon.b.smithi161.stdout: return _run_code(code, main_globals, None, 2024-06-01T02:02:06.450 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-06-01T02:02:06.450 INFO:journalctl@ceph.mon.b.smithi161.stdout: exec(code, run_globals) 2024-06-01T02:02:06.451 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5618, in 2024-06-01T02:02:06.451 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5606, in main 2024-06-01T02:02:06.451 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3096, in command_deploy_from 2024-06-01T02:02:06.451 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3131, in _common_deploy 2024-06-01T02:02:06.451 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3151, in _deploy_daemon_container 2024-06-01T02:02:06.451 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 1123, in deploy_daemon 2024-06-01T02:02:06.452 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-06-01T02:02:06.452 INFO:journalctl@ceph.mon.b.smithi161.stdout: RuntimeError: Failed command: systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb: Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:02:06.452 INFO:journalctl@ceph.mon.b.smithi161.stdout: See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:02:06.452 INFO:journalctl@ceph.mon.b.smithi161.stdout: Traceback (most recent call last): 2024-06-01T02:02:06.452 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1114, in _check_daemons 2024-06-01T02:02:06.452 INFO:journalctl@ceph.mon.b.smithi161.stdout: self.mgr._daemon_action(daemon_spec, action=action) 2024-06-01T02:02:06.452 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 2457, in _daemon_action 2024-06-01T02:02:06.453 INFO:journalctl@ceph.mon.b.smithi161.stdout: CephadmServe(self)._create_daemon(daemon_spec, reconfig=(action == 'reconfig'))) 2024-06-01T02:02:06.453 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 779, in wait_async 2024-06-01T02:02:06.453 INFO:journalctl@ceph.mon.b.smithi161.stdout: return self.event_loop.get_result(coro, timeout) 2024-06-01T02:02:06.453 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/usr/share/ceph/mgr/cephadm/ssh.py", line 136, in get_result 2024-06-01T02:02:06.453 INFO:journalctl@ceph.mon.b.smithi161.stdout: return future.result(timeout) 2024-06-01T02:02:06.453 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 432, in result 2024-06-01T02:02:06.454 INFO:journalctl@ceph.mon.b.smithi161.stdout: return self.__get_result() 2024-06-01T02:02:06.454 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result 2024-06-01T02:02:06.454 INFO:journalctl@ceph.mon.b.smithi161.stdout: raise self._exception 2024-06-01T02:02:06.454 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1377, in _create_daemon 2024-06-01T02:02:06.454 INFO:journalctl@ceph.mon.b.smithi161.stdout: use_current_daemon_image=reconfig, 2024-06-01T02:02:06.454 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1692, in _run_cephadm 2024-06-01T02:02:06.455 INFO:journalctl@ceph.mon.b.smithi161.stdout: f'cephadm exited with an error code: {code}, stderr: {err}') 2024-06-01T02:02:06.455 INFO:journalctl@ceph.mon.b.smithi161.stdout: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi116.gjuxzb ... 2024-06-01T02:02:06.455 INFO:journalctl@ceph.mon.b.smithi161.stdout: Creating ceph-iscsi config... 2024-06-01T02:02:06.455 INFO:journalctl@ceph.mon.b.smithi161.stdout: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/iscsi-gateway.cfg 2024-06-01T02:02:06.456 INFO:journalctl@ceph.mon.b.smithi161.stdout: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/tcmu-runner-entrypoint.sh 2024-06-01T02:02:06.456 INFO:journalctl@ceph.mon.b.smithi161.stdout: Non-zero exit code 1 from systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb 2024-06-01T02:02:06.456 INFO:journalctl@ceph.mon.b.smithi161.stdout: systemctl: stderr Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:02:06.456 INFO:journalctl@ceph.mon.b.smithi161.stdout: systemctl: stderr See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:02:06.456 INFO:journalctl@ceph.mon.b.smithi161.stdout: Traceback (most recent call last): 2024-06-01T02:02:06.456 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-06-01T02:02:06.456 INFO:journalctl@ceph.mon.b.smithi161.stdout: return _run_code(code, main_globals, None, 2024-06-01T02:02:06.457 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-06-01T02:02:06.457 INFO:journalctl@ceph.mon.b.smithi161.stdout: exec(code, run_globals) 2024-06-01T02:02:06.457 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5618, in 2024-06-01T02:02:06.457 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5606, in main 2024-06-01T02:02:06.457 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3096, in command_deploy_from 2024-06-01T02:02:06.457 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3131, in _common_deploy 2024-06-01T02:02:06.458 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3151, in _deploy_daemon_container 2024-06-01T02:02:06.458 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 1123, in deploy_daemon 2024-06-01T02:02:06.458 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-06-01T02:02:06.458 INFO:journalctl@ceph.mon.b.smithi161.stdout: RuntimeError: Failed command: systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb: Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:02:06.458 INFO:journalctl@ceph.mon.b.smithi161.stdout: See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:02:06.458 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:06 smithi161 ceph-mon[26055]: Reconfiguring daemon agent.smithi161 on smithi161 2024-06-01T02:02:06.459 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:06 smithi161 ceph-mon[26055]: Deploying cephadm binary to smithi161 2024-06-01T02:02:06.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:06 smithi116 ceph-mon[25082]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:02:06.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:06 smithi116 ceph-mon[25082]: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi116.gjuxzb ... 2024-06-01T02:02:06.794 INFO:journalctl@ceph.mon.a.smithi116.stdout: Creating ceph-iscsi config... 2024-06-01T02:02:06.794 INFO:journalctl@ceph.mon.a.smithi116.stdout: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/iscsi-gateway.cfg 2024-06-01T02:02:06.794 INFO:journalctl@ceph.mon.a.smithi116.stdout: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/tcmu-runner-entrypoint.sh 2024-06-01T02:02:06.794 INFO:journalctl@ceph.mon.a.smithi116.stdout: Non-zero exit code 1 from systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb 2024-06-01T02:02:06.794 INFO:journalctl@ceph.mon.a.smithi116.stdout: systemctl: stderr Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:02:06.794 INFO:journalctl@ceph.mon.a.smithi116.stdout: systemctl: stderr See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:02:06.794 INFO:journalctl@ceph.mon.a.smithi116.stdout: Traceback (most recent call last): 2024-06-01T02:02:06.794 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-06-01T02:02:06.795 INFO:journalctl@ceph.mon.a.smithi116.stdout: return _run_code(code, main_globals, None, 2024-06-01T02:02:06.795 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-06-01T02:02:06.795 INFO:journalctl@ceph.mon.a.smithi116.stdout: exec(code, run_globals) 2024-06-01T02:02:06.795 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5618, in 2024-06-01T02:02:06.795 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5606, in main 2024-06-01T02:02:06.795 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3096, in command_deploy_from 2024-06-01T02:02:06.795 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3131, in _common_deploy 2024-06-01T02:02:06.795 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3151, in _deploy_daemon_container 2024-06-01T02:02:06.795 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 1123, in deploy_daemon 2024-06-01T02:02:06.795 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-06-01T02:02:06.795 INFO:journalctl@ceph.mon.a.smithi116.stdout: RuntimeError: Failed command: systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb: Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:02:06.795 INFO:journalctl@ceph.mon.a.smithi116.stdout: See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:02:06.795 INFO:journalctl@ceph.mon.a.smithi116.stdout: Traceback (most recent call last): 2024-06-01T02:02:06.795 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1114, in _check_daemons 2024-06-01T02:02:06.796 INFO:journalctl@ceph.mon.a.smithi116.stdout: self.mgr._daemon_action(daemon_spec, action=action) 2024-06-01T02:02:06.796 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 2457, in _daemon_action 2024-06-01T02:02:06.796 INFO:journalctl@ceph.mon.a.smithi116.stdout: CephadmServe(self)._create_daemon(daemon_spec, reconfig=(action == 'reconfig'))) 2024-06-01T02:02:06.796 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 779, in wait_async 2024-06-01T02:02:06.796 INFO:journalctl@ceph.mon.a.smithi116.stdout: return self.event_loop.get_result(coro, timeout) 2024-06-01T02:02:06.796 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/usr/share/ceph/mgr/cephadm/ssh.py", line 136, in get_result 2024-06-01T02:02:06.796 INFO:journalctl@ceph.mon.a.smithi116.stdout: return future.result(timeout) 2024-06-01T02:02:06.796 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 432, in result 2024-06-01T02:02:06.796 INFO:journalctl@ceph.mon.a.smithi116.stdout: return self.__get_result() 2024-06-01T02:02:06.796 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result 2024-06-01T02:02:06.796 INFO:journalctl@ceph.mon.a.smithi116.stdout: raise self._exception 2024-06-01T02:02:06.796 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1377, in _create_daemon 2024-06-01T02:02:06.796 INFO:journalctl@ceph.mon.a.smithi116.stdout: use_current_daemon_image=reconfig, 2024-06-01T02:02:06.796 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1692, in _run_cephadm 2024-06-01T02:02:06.797 INFO:journalctl@ceph.mon.a.smithi116.stdout: f'cephadm exited with an error code: {code}, stderr: {err}') 2024-06-01T02:02:06.797 INFO:journalctl@ceph.mon.a.smithi116.stdout: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi116.gjuxzb ... 2024-06-01T02:02:06.797 INFO:journalctl@ceph.mon.a.smithi116.stdout: Creating ceph-iscsi config... 2024-06-01T02:02:06.797 INFO:journalctl@ceph.mon.a.smithi116.stdout: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/iscsi-gateway.cfg 2024-06-01T02:02:06.797 INFO:journalctl@ceph.mon.a.smithi116.stdout: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/tcmu-runner-entrypoint.sh 2024-06-01T02:02:06.797 INFO:journalctl@ceph.mon.a.smithi116.stdout: Non-zero exit code 1 from systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb 2024-06-01T02:02:06.797 INFO:journalctl@ceph.mon.a.smithi116.stdout: systemctl: stderr Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:02:06.797 INFO:journalctl@ceph.mon.a.smithi116.stdout: systemctl: stderr See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:02:06.797 INFO:journalctl@ceph.mon.a.smithi116.stdout: Traceback (most recent call last): 2024-06-01T02:02:06.797 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-06-01T02:02:06.797 INFO:journalctl@ceph.mon.a.smithi116.stdout: return _run_code(code, main_globals, None, 2024-06-01T02:02:06.797 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-06-01T02:02:06.797 INFO:journalctl@ceph.mon.a.smithi116.stdout: exec(code, run_globals) 2024-06-01T02:02:06.797 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5618, in 2024-06-01T02:02:06.798 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5606, in main 2024-06-01T02:02:06.798 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3096, in command_deploy_from 2024-06-01T02:02:06.798 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3131, in _common_deploy 2024-06-01T02:02:06.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:06 smithi116 ceph-mon[29958]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:02:06.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:06 smithi116 ceph-mon[29958]: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi116.gjuxzb ... 2024-06-01T02:02:06.798 INFO:journalctl@ceph.mon.c.smithi116.stdout: Creating ceph-iscsi config... 2024-06-01T02:02:06.798 INFO:journalctl@ceph.mon.c.smithi116.stdout: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/iscsi-gateway.cfg 2024-06-01T02:02:06.798 INFO:journalctl@ceph.mon.c.smithi116.stdout: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/tcmu-runner-entrypoint.sh 2024-06-01T02:02:06.799 INFO:journalctl@ceph.mon.c.smithi116.stdout: Non-zero exit code 1 from systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb 2024-06-01T02:02:06.799 INFO:journalctl@ceph.mon.c.smithi116.stdout: systemctl: stderr Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:02:06.799 INFO:journalctl@ceph.mon.c.smithi116.stdout: systemctl: stderr See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:02:06.799 INFO:journalctl@ceph.mon.c.smithi116.stdout: Traceback (most recent call last): 2024-06-01T02:02:06.799 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-06-01T02:02:06.799 INFO:journalctl@ceph.mon.c.smithi116.stdout: return _run_code(code, main_globals, None, 2024-06-01T02:02:06.799 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-06-01T02:02:06.799 INFO:journalctl@ceph.mon.c.smithi116.stdout: exec(code, run_globals) 2024-06-01T02:02:06.799 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5618, in 2024-06-01T02:02:06.799 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5606, in main 2024-06-01T02:02:06.799 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3096, in command_deploy_from 2024-06-01T02:02:06.799 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3131, in _common_deploy 2024-06-01T02:02:06.800 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3151, in _deploy_daemon_container 2024-06-01T02:02:06.800 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 1123, in deploy_daemon 2024-06-01T02:02:06.800 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-06-01T02:02:06.800 INFO:journalctl@ceph.mon.c.smithi116.stdout: RuntimeError: Failed command: systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb: Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:02:06.800 INFO:journalctl@ceph.mon.c.smithi116.stdout: See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:02:06.800 INFO:journalctl@ceph.mon.c.smithi116.stdout: Traceback (most recent call last): 2024-06-01T02:02:06.800 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1114, in _check_daemons 2024-06-01T02:02:06.800 INFO:journalctl@ceph.mon.c.smithi116.stdout: self.mgr._daemon_action(daemon_spec, action=action) 2024-06-01T02:02:06.800 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 2457, in _daemon_action 2024-06-01T02:02:06.800 INFO:journalctl@ceph.mon.c.smithi116.stdout: CephadmServe(self)._create_daemon(daemon_spec, reconfig=(action == 'reconfig'))) 2024-06-01T02:02:06.800 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 779, in wait_async 2024-06-01T02:02:06.800 INFO:journalctl@ceph.mon.c.smithi116.stdout: return self.event_loop.get_result(coro, timeout) 2024-06-01T02:02:06.800 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/usr/share/ceph/mgr/cephadm/ssh.py", line 136, in get_result 2024-06-01T02:02:06.800 INFO:journalctl@ceph.mon.c.smithi116.stdout: return future.result(timeout) 2024-06-01T02:02:06.801 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 432, in result 2024-06-01T02:02:06.801 INFO:journalctl@ceph.mon.c.smithi116.stdout: return self.__get_result() 2024-06-01T02:02:06.801 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result 2024-06-01T02:02:06.801 INFO:journalctl@ceph.mon.c.smithi116.stdout: raise self._exception 2024-06-01T02:02:06.801 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1377, in _create_daemon 2024-06-01T02:02:06.801 INFO:journalctl@ceph.mon.c.smithi116.stdout: use_current_daemon_image=reconfig, 2024-06-01T02:02:06.801 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1692, in _run_cephadm 2024-06-01T02:02:06.801 INFO:journalctl@ceph.mon.c.smithi116.stdout: f'cephadm exited with an error code: {code}, stderr: {err}') 2024-06-01T02:02:06.801 INFO:journalctl@ceph.mon.c.smithi116.stdout: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi116.gjuxzb ... 2024-06-01T02:02:06.801 INFO:journalctl@ceph.mon.c.smithi116.stdout: Creating ceph-iscsi config... 2024-06-01T02:02:06.801 INFO:journalctl@ceph.mon.c.smithi116.stdout: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/iscsi-gateway.cfg 2024-06-01T02:02:06.801 INFO:journalctl@ceph.mon.c.smithi116.stdout: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/tcmu-runner-entrypoint.sh 2024-06-01T02:02:06.801 INFO:journalctl@ceph.mon.c.smithi116.stdout: Non-zero exit code 1 from systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb 2024-06-01T02:02:06.801 INFO:journalctl@ceph.mon.c.smithi116.stdout: systemctl: stderr Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:02:06.802 INFO:journalctl@ceph.mon.c.smithi116.stdout: systemctl: stderr See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:02:06.802 INFO:journalctl@ceph.mon.c.smithi116.stdout: Traceback (most recent call last): 2024-06-01T02:02:06.802 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-06-01T02:02:06.802 INFO:journalctl@ceph.mon.c.smithi116.stdout: return _run_code(code, main_globals, None, 2024-06-01T02:02:06.802 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-06-01T02:02:06.802 INFO:journalctl@ceph.mon.c.smithi116.stdout: exec(code, run_globals) 2024-06-01T02:02:06.802 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5618, in 2024-06-01T02:02:06.802 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5606, in main 2024-06-01T02:02:06.802 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3096, in command_deploy_from 2024-06-01T02:02:06.802 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3131, in _common_deploy 2024-06-01T02:02:06.802 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3151, in _deploy_daemon_container 2024-06-01T02:02:06.803 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 1123, in deploy_daemon 2024-06-01T02:02:06.803 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-06-01T02:02:06.803 INFO:journalctl@ceph.mon.a.smithi116.stdout: RuntimeError: Failed command: systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb: Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:02:06.803 INFO:journalctl@ceph.mon.a.smithi116.stdout: See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:02:06.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:06 smithi116 ceph-mon[25082]: Reconfiguring daemon agent.smithi161 on smithi161 2024-06-01T02:02:06.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:06 smithi116 ceph-mon[25082]: Deploying cephadm binary to smithi161 2024-06-01T02:02:06.803 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3151, in _deploy_daemon_container 2024-06-01T02:02:06.803 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 1123, in deploy_daemon 2024-06-01T02:02:06.804 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-06-01T02:02:06.804 INFO:journalctl@ceph.mon.c.smithi116.stdout: RuntimeError: Failed command: systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb: Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:02:06.804 INFO:journalctl@ceph.mon.c.smithi116.stdout: See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:02:06.804 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:06 smithi116 ceph-mon[29958]: Reconfiguring daemon agent.smithi161 on smithi161 2024-06-01T02:02:06.804 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:06 smithi116 ceph-mon[29958]: Deploying cephadm binary to smithi161 2024-06-01T02:02:08.647 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:08 smithi161 ceph-mon[26055]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:02:08.648 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:08 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:08.648 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:08 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:08.648 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:08 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:08.793 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:08 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[25312]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:02:08] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-06-01T02:02:08.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:08 smithi116 ceph-mon[25082]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:02:08.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:08 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:08.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:08 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:08.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:08 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:08.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:08 smithi116 ceph-mon[29958]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:02:08.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:08 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:08.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:08 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:08.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:08 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:09.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:09 smithi161 ceph-mon[26055]: Reconfiguring prometheus.a (dependencies changed)... 2024-06-01T02:02:09.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:09 smithi161 ceph-mon[26055]: Reconfiguring daemon prometheus.a on smithi161 2024-06-01T02:02:09.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:09 smithi116 ceph-mon[29958]: Reconfiguring prometheus.a (dependencies changed)... 2024-06-01T02:02:09.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:09 smithi116 ceph-mon[29958]: Reconfiguring daemon prometheus.a on smithi161 2024-06-01T02:02:09.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:09 smithi116 ceph-mon[25082]: Reconfiguring prometheus.a (dependencies changed)... 2024-06-01T02:02:09.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:09 smithi116 ceph-mon[25082]: Reconfiguring daemon prometheus.a on smithi161 2024-06-01T02:02:10.512 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:10 smithi116 ceph-mon[25082]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:02:10.512 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:10 smithi116 ceph-mon[29958]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:02:10.636 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:10 smithi161 ceph-mon[26055]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:02:11.156 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:02:10 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[64300]: level=info ts=2024-06-01T02:02:10.832Z caller=cluster.go:688 component=cluster msg="gossip settled; proceeding" elapsed=10.003186124s 2024-06-01T02:02:12.030 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:11 smithi161 systemd[1]: Stopping Ceph prometheus.a for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:02:12.030 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:11 smithi161 bash[54681]: Error: no container with name or ID "ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus.a" found: no such container 2024-06-01T02:02:12.346 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:12 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T02:02:12.034Z caller=main.go:775 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-06-01T02:02:12.346 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:12 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T02:02:12.034Z caller=main.go:798 level=info msg="Stopping scrape discovery manager..." 2024-06-01T02:02:12.346 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:12 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T02:02:12.034Z caller=main.go:812 level=info msg="Stopping notify discovery manager..." 2024-06-01T02:02:12.346 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:12 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T02:02:12.034Z caller=main.go:834 level=info msg="Stopping scrape manager..." 2024-06-01T02:02:12.347 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:12 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T02:02:12.034Z caller=main.go:794 level=info msg="Scrape discovery manager stopped" 2024-06-01T02:02:12.347 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:12 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T02:02:12.034Z caller=main.go:808 level=info msg="Notify discovery manager stopped" 2024-06-01T02:02:12.347 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:12 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T02:02:12.034Z caller=manager.go:945 level=info component="rule manager" msg="Stopping rule manager..." 2024-06-01T02:02:12.347 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:12 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T02:02:12.034Z caller=main.go:828 level=info msg="Scrape manager stopped" 2024-06-01T02:02:12.347 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:12 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T02:02:12.034Z caller=manager.go:955 level=info component="rule manager" msg="Rule manager stopped" 2024-06-01T02:02:12.347 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:12 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T02:02:12.035Z caller=notifier.go:600 level=info component=notifier msg="Stopping notification manager..." 2024-06-01T02:02:12.347 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:12 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T02:02:12.035Z caller=main.go:1054 level=info msg="Notifier manager stopped" 2024-06-01T02:02:12.347 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:12 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[45448]: ts=2024-06-01T02:02:12.035Z caller=main.go:1066 level=info msg="See you next time!" 2024-06-01T02:02:12.347 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:12 smithi161 podman[54692]: 2024-06-01 02:02:12.04457165 +0000 UTC m=+0.124458170 container died bea6bf7e708305690304e13f81d48feca6103d9a6752f57a0a7b4a21ab34c93a (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-01T02:02:12.347 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:12 smithi161 podman[54692]: 2024-06-01 02:02:12.350535185 +0000 UTC m=+0.430421704 container cleanup bea6bf7e708305690304e13f81d48feca6103d9a6752f57a0a7b4a21ab34c93a (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-01T02:02:12.347 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:12 smithi161 bash[54692]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a 2024-06-01T02:02:12.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:12 smithi116 ceph-mon[29958]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:02:12.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:12 smithi116 ceph-mon[25082]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:02:12.602 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:12 smithi161 ceph-mon[26055]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:02:12.602 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:12 smithi161 bash[54732]: Error: no container with name or ID "ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus.a" found: no such container 2024-06-01T02:02:12.602 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:12 smithi161 podman[54714]: 2024-06-01 02:02:12.584883494 +0000 UTC m=+0.539571565 container remove bea6bf7e708305690304e13f81d48feca6103d9a6752f57a0a7b4a21ab34c93a (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-01T02:02:12.602 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:12 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-06-01T02:02:12.602 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:12 smithi161 systemd[1]: Stopped Ceph prometheus.a for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:02:12.602 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:12 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@prometheus.a.service: Consumed 1.420s CPU time. 2024-06-01T02:02:12.874 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:12 smithi161 systemd[1]: Starting Ceph prometheus.a for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:02:13.645 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:13 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:02:13.744 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:13 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:02:13.745 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:13 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:02:14.335 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:14 smithi161 podman[54905]: 2024-06-01 02:02:13.99158262 +0000 UTC m=+0.648839559 image pull 514e6a882f6e74806a5856468489eeff8d7106095557578da96935e4d0ba4d9d quay.io/prometheus/prometheus:v2.33.4 2024-06-01T02:02:14.335 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:14 smithi161 podman[54905]: 2024-06-01 02:02:14.093452807 +0000 UTC m=+0.750709722 container create 1267ec8fa4c91e115570028fadb0074ab5429234244664badbf3a05de353bc95 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-01T02:02:14.335 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:14 smithi161 podman[54905]: 2024-06-01 02:02:14.312275606 +0000 UTC m=+0.969532525 container init 1267ec8fa4c91e115570028fadb0074ab5429234244664badbf3a05de353bc95 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-01T02:02:14.335 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:14 smithi161 podman[54905]: 2024-06-01 02:02:14.315133318 +0000 UTC m=+0.972390242 container start 1267ec8fa4c91e115570028fadb0074ab5429234244664badbf3a05de353bc95 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-01T02:02:14.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:14 smithi161 ceph-mon[26055]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 426 B/s rd, 0 op/s 2024-06-01T02:02:14.587 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:14.339Z caller=main.go:475 level=info msg="No time or size retention was set so using the default time retention" duration=15d 2024-06-01T02:02:14.587 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:14.339Z caller=main.go:512 level=info msg="Starting Prometheus" version="(version=2.33.4, branch=HEAD, revision=83032011a5d3e6102624fe58241a374a7201fee8)" 2024-06-01T02:02:14.588 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:14.339Z caller=main.go:517 level=info build_context="(go=go1.17.7, user=root@d13bf69e7be8, date=20220222-16:51:28)" 2024-06-01T02:02:14.588 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:14.339Z caller=main.go:518 level=info host_details="(Linux 5.14.0-452.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024 x86_64 smithi161 (none))" 2024-06-01T02:02:14.588 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:14.339Z caller=main.go:519 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-06-01T02:02:14.588 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:14.339Z caller=main.go:520 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-06-01T02:02:14.588 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:14.340Z caller=web.go:570 level=info component=web msg="Start listening for connections" address=:9095 2024-06-01T02:02:14.588 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:14.341Z caller=main.go:923 level=info msg="Starting TSDB ..." 2024-06-01T02:02:14.588 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:14.342Z caller=tls_config.go:195 level=info component=web msg="TLS is disabled." http2=false 2024-06-01T02:02:14.588 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:14.344Z caller=head.go:493 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-06-01T02:02:14.588 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:14.345Z caller=head.go:527 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.340579ms 2024-06-01T02:02:14.588 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:14.345Z caller=head.go:533 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-06-01T02:02:14.588 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:14.347Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=2 2024-06-01T02:02:14.589 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:14 smithi161 bash[54905]: 1267ec8fa4c91e115570028fadb0074ab5429234244664badbf3a05de353bc95 2024-06-01T02:02:14.589 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:14 smithi161 systemd[1]: Started Ceph prometheus.a for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:02:14.634 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:14 smithi116 ceph-mon[25082]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 426 B/s rd, 0 op/s 2024-06-01T02:02:14.634 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:14 smithi116 ceph-mon[29958]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 426 B/s rd, 0 op/s 2024-06-01T02:02:14.841 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:14.810Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=2 2024-06-01T02:02:14.842 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:14.810Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=2 maxSegment=2 2024-06-01T02:02:14.842 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:14.810Z caller=head.go:610 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=34.156µs wal_replay_duration=464.687264ms total_replay_duration=466.080129ms 2024-06-01T02:02:14.842 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:14.815Z caller=main.go:944 level=info fs_type=XFS_SUPER_MAGIC 2024-06-01T02:02:14.842 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:14.815Z caller=main.go:947 level=info msg="TSDB started" 2024-06-01T02:02:14.842 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:14.815Z caller=main.go:1128 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-06-01T02:02:14.842 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:14.845Z caller=main.go:1165 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=30.556089ms db_storage=759ns remote_storage=1.681µs web_handler=406ns query_engine=1.149µs scrape=566.069µs scrape_sd=163.998µs notify=17.542µs notify_sd=21.655µs rules=29.130106ms 2024-06-01T02:02:14.842 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:14.845Z caller=main.go:896 level=info msg="Server is ready to receive web requests." 2024-06-01T02:02:15.760 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:15 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:15.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:15 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:15.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:15 smithi116 ceph-mon[25082]: Reconfiguring node-exporter.b (dependencies changed)... 2024-06-01T02:02:15.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:15 smithi116 ceph-mon[25082]: Reconfiguring daemon node-exporter.b on smithi161 2024-06-01T02:02:15.761 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:15 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:15.761 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:15 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:15.762 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:15 smithi116 ceph-mon[29958]: Reconfiguring node-exporter.b (dependencies changed)... 2024-06-01T02:02:15.762 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:15 smithi116 ceph-mon[29958]: Reconfiguring daemon node-exporter.b on smithi161 2024-06-01T02:02:15.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:15 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:15.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:15 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:15.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:15 smithi161 ceph-mon[26055]: Reconfiguring node-exporter.b (dependencies changed)... 2024-06-01T02:02:15.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:15 smithi161 ceph-mon[26055]: Reconfiguring daemon node-exporter.b on smithi161 2024-06-01T02:02:16.539 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:16 smithi161 systemd[1]: Stopping Ceph node-exporter.b for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:02:16.539 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:16 smithi161 ceph-mon[26055]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 426 B/s rd, 0 op/s 2024-06-01T02:02:16.540 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:16 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1491895736' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-01T02:02:16.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:16 smithi116 ceph-mon[25082]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 426 B/s rd, 0 op/s 2024-06-01T02:02:16.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:16 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1491895736' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-01T02:02:16.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:16 smithi116 ceph-mon[29958]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 426 B/s rd, 0 op/s 2024-06-01T02:02:16.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:16 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1491895736' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-01T02:02:16.944 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:16 smithi161 bash[55328]: Error: no container with name or ID "ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter.b" found: no such container 2024-06-01T02:02:17.442 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:17 smithi161 podman[55363]: 2024-06-01 02:02:17.141286893 +0000 UTC m=+0.471612833 container died 4456eaae965b95cef5ae97c9c41d5e27da0aaac4ad51e214d2f8cd4a95ecda12 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-06-01T02:02:17.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:17 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/3970602698' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/189859058"}]: dispatch 2024-06-01T02:02:17.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:17 smithi116 ceph-mon[25082]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/189859058"}]: dispatch 2024-06-01T02:02:17.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:17 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/3970602698' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/189859058"}]: dispatch 2024-06-01T02:02:17.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:17 smithi116 ceph-mon[29958]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/189859058"}]: dispatch 2024-06-01T02:02:17.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:17 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/3970602698' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/189859058"}]: dispatch 2024-06-01T02:02:17.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:17 smithi161 ceph-mon[26055]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/189859058"}]: dispatch 2024-06-01T02:02:17.695 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:17 smithi161 podman[55363]: 2024-06-01 02:02:17.446964641 +0000 UTC m=+0.777290581 container cleanup 4456eaae965b95cef5ae97c9c41d5e27da0aaac4ad51e214d2f8cd4a95ecda12 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-06-01T02:02:17.695 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:17 smithi161 bash[55363]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b 2024-06-01T02:02:17.695 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:17 smithi161 bash[55430]: Error: no container with name or ID "ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter.b" found: no such container 2024-06-01T02:02:17.695 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:17 smithi161 podman[55407]: 2024-06-01 02:02:17.665200236 +0000 UTC m=+0.527587672 container remove 4456eaae965b95cef5ae97c9c41d5e27da0aaac4ad51e214d2f8cd4a95ecda12 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-06-01T02:02:17.695 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:17 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-06-01T02:02:17.695 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:17 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-06-01T02:02:17.695 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:17 smithi161 systemd[1]: Stopped Ceph node-exporter.b for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:02:17.696 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:17 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@node-exporter.b.service: Consumed 2.776s CPU time. 2024-06-01T02:02:17.696 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:17 smithi161 systemd[1]: Starting Ceph node-exporter.b for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:02:18.751 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 podman[55568]: 2024-06-01 02:02:18.429281402 +0000 UTC m=+0.018327250 image pull 1dbe0e931976487e20e5cfb272087e08a9779c88fd5e9617ed7042dd9751ec26 quay.io/prometheus/node-exporter:v1.3.1 2024-06-01T02:02:18.751 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 podman[55568]: 2024-06-01 02:02:18.544856575 +0000 UTC m=+0.133902424 container create 6e66182f5a1afcc0d2c6acbcdd5f7a12b8a739c4ebc2cca9d46319cf2ace23fa (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-06-01T02:02:18.751 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-mon[26055]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:18.752 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-mon[26055]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/189859058"}]': finished 2024-06-01T02:02:18.752 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-mon[26055]: osdmap e98: 8 total, 8 up, 8 in 2024-06-01T02:02:18.752 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/2785801205' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3804553129"}]: dispatch 2024-06-01T02:02:18.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:18 smithi116 ceph-mon[25082]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:18.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:18 smithi116 ceph-mon[25082]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/189859058"}]': finished 2024-06-01T02:02:18.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:18 smithi116 ceph-mon[25082]: osdmap e98: 8 total, 8 up, 8 in 2024-06-01T02:02:18.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:18 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/2785801205' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3804553129"}]: dispatch 2024-06-01T02:02:18.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:18 smithi116 ceph-mon[29958]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:18.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:18 smithi116 ceph-mon[29958]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/189859058"}]': finished 2024-06-01T02:02:18.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:18 smithi116 ceph-mon[29958]: osdmap e98: 8 total, 8 up, 8 in 2024-06-01T02:02:18.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:18 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/2785801205' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3804553129"}]: dispatch 2024-06-01T02:02:19.195 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 podman[55568]: 2024-06-01 02:02:18.755628059 +0000 UTC m=+0.344673906 container init 6e66182f5a1afcc0d2c6acbcdd5f7a12b8a739c4ebc2cca9d46319cf2ace23fa (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-06-01T02:02:19.195 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 podman[55568]: 2024-06-01 02:02:18.758760677 +0000 UTC m=+0.347806525 container start 6e66182f5a1afcc0d2c6acbcdd5f7a12b8a739c4ebc2cca9d46319cf2ace23fa (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-06-01T02:02:19.195 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.762Z caller=node_exporter.go:182 level=info msg="Starting node_exporter" version="(version=1.3.1, branch=HEAD, revision=a2321e7b940ddcff26873612bccdf7cd4c42b6b6)" 2024-06-01T02:02:19.195 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.762Z caller=node_exporter.go:183 level=info msg="Build context" build_context="(go=go1.17.3, user=root@243aafa5525c, date=20211205-11:09:49)" 2024-06-01T02:02:19.195 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+)($|/) 2024-06-01T02:02:19.195 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-06-01T02:02:19.196 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:108 level=info msg="Enabled collectors" 2024-06-01T02:02:19.196 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=arp 2024-06-01T02:02:19.196 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=bcache 2024-06-01T02:02:19.196 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=bonding 2024-06-01T02:02:19.196 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=btrfs 2024-06-01T02:02:19.196 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=conntrack 2024-06-01T02:02:19.196 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=cpu 2024-06-01T02:02:19.196 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=cpufreq 2024-06-01T02:02:19.196 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=diskstats 2024-06-01T02:02:19.196 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=dmi 2024-06-01T02:02:19.196 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=edac 2024-06-01T02:02:19.197 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=entropy 2024-06-01T02:02:19.197 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=fibrechannel 2024-06-01T02:02:19.197 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=filefd 2024-06-01T02:02:19.197 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=filesystem 2024-06-01T02:02:19.197 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=hwmon 2024-06-01T02:02:19.197 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=infiniband 2024-06-01T02:02:19.197 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=ipvs 2024-06-01T02:02:19.198 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=loadavg 2024-06-01T02:02:19.198 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=mdadm 2024-06-01T02:02:19.198 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=meminfo 2024-06-01T02:02:19.198 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=netclass 2024-06-01T02:02:19.198 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=netdev 2024-06-01T02:02:19.198 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=netstat 2024-06-01T02:02:19.198 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=nfs 2024-06-01T02:02:19.198 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=nfsd 2024-06-01T02:02:19.198 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=nvme 2024-06-01T02:02:19.198 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=os 2024-06-01T02:02:19.198 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=powersupplyclass 2024-06-01T02:02:19.198 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=pressure 2024-06-01T02:02:19.198 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=rapl 2024-06-01T02:02:19.199 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=schedstat 2024-06-01T02:02:19.199 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=sockstat 2024-06-01T02:02:19.199 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=softnet 2024-06-01T02:02:19.199 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=stat 2024-06-01T02:02:19.199 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=tapestats 2024-06-01T02:02:19.199 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=textfile 2024-06-01T02:02:19.199 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=thermal_zone 2024-06-01T02:02:19.199 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=time 2024-06-01T02:02:19.199 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=udp_queues 2024-06-01T02:02:19.199 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=uname 2024-06-01T02:02:19.199 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=vmstat 2024-06-01T02:02:19.199 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=xfs 2024-06-01T02:02:19.199 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:115 level=info collector=zfs 2024-06-01T02:02:19.199 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=node_exporter.go:199 level=info msg="Listening on" address=:9100 2024-06-01T02:02:19.200 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b[55596]: ts=2024-06-01T02:02:18.763Z caller=tls_config.go:195 level=info msg="TLS is disabled." http2=false 2024-06-01T02:02:19.200 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 bash[55568]: 6e66182f5a1afcc0d2c6acbcdd5f7a12b8a739c4ebc2cca9d46319cf2ace23fa 2024-06-01T02:02:19.200 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:02:18 smithi161 systemd[1]: Started Ceph node-exporter.b for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:02:19.750 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:19 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/2785801205' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3804553129"}]': finished 2024-06-01T02:02:19.751 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:19 smithi161 ceph-mon[26055]: osdmap e99: 8 total, 8 up, 8 in 2024-06-01T02:02:19.751 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:19 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1204554059' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/2988728965"}]: dispatch 2024-06-01T02:02:19.751 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:19 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:19.751 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:19 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:19.751 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:19 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:19.751 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:19 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:19.751 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:19 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-01T02:02:19.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:19 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/2785801205' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3804553129"}]': finished 2024-06-01T02:02:19.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:19 smithi116 ceph-mon[25082]: osdmap e99: 8 total, 8 up, 8 in 2024-06-01T02:02:19.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:19 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1204554059' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/2988728965"}]: dispatch 2024-06-01T02:02:19.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:19 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:19.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:19 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:19.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:19 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:19.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:19 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:19.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:19 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-01T02:02:19.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:19 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/2785801205' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3804553129"}]': finished 2024-06-01T02:02:19.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:19 smithi116 ceph-mon[29958]: osdmap e99: 8 total, 8 up, 8 in 2024-06-01T02:02:19.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:19 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1204554059' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/2988728965"}]: dispatch 2024-06-01T02:02:19.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:19 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:19.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:19 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:19.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:19 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:19.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:19 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:19.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:19 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-01T02:02:20.770 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:20 smithi161 ceph-mon[26055]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:20.770 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:20 smithi161 ceph-mon[26055]: Reconfiguring grafana.a (dependencies changed)... 2024-06-01T02:02:20.770 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:20 smithi161 ceph-mon[26055]: Regenerating cephadm self-signed grafana TLS certificates 2024-06-01T02:02:20.770 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:20 smithi161 ceph-mon[26055]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-01T02:02:20.770 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:20 smithi161 ceph-mon[26055]: Reconfiguring daemon grafana.a on smithi161 2024-06-01T02:02:20.770 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:20 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1204554059' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/2988728965"}]': finished 2024-06-01T02:02:20.770 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:20 smithi161 ceph-mon[26055]: osdmap e100: 8 total, 8 up, 8 in 2024-06-01T02:02:20.770 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:20 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/4102838325' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/189859058"}]: dispatch 2024-06-01T02:02:20.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:20 smithi116 ceph-mon[25082]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:20.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:20 smithi116 ceph-mon[25082]: Reconfiguring grafana.a (dependencies changed)... 2024-06-01T02:02:20.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:20 smithi116 ceph-mon[25082]: Regenerating cephadm self-signed grafana TLS certificates 2024-06-01T02:02:20.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:20 smithi116 ceph-mon[25082]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-01T02:02:20.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:20 smithi116 ceph-mon[25082]: Reconfiguring daemon grafana.a on smithi161 2024-06-01T02:02:20.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:20 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1204554059' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/2988728965"}]': finished 2024-06-01T02:02:20.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:20 smithi116 ceph-mon[25082]: osdmap e100: 8 total, 8 up, 8 in 2024-06-01T02:02:20.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:20 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/4102838325' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/189859058"}]: dispatch 2024-06-01T02:02:20.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:20 smithi116 ceph-mon[29958]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:20.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:20 smithi116 ceph-mon[29958]: Reconfiguring grafana.a (dependencies changed)... 2024-06-01T02:02:20.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:20 smithi116 ceph-mon[29958]: Regenerating cephadm self-signed grafana TLS certificates 2024-06-01T02:02:20.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:20 smithi116 ceph-mon[29958]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-01T02:02:20.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:20 smithi116 ceph-mon[29958]: Reconfiguring daemon grafana.a on smithi161 2024-06-01T02:02:20.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:20 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1204554059' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/2988728965"}]': finished 2024-06-01T02:02:20.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:20 smithi116 ceph-mon[29958]: osdmap e100: 8 total, 8 up, 8 in 2024-06-01T02:02:20.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:20 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/4102838325' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/189859058"}]: dispatch 2024-06-01T02:02:21.793 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:21 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/4102838325' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/189859058"}]': finished 2024-06-01T02:02:21.793 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:21 smithi161 ceph-mon[26055]: osdmap e101: 8 total, 8 up, 8 in 2024-06-01T02:02:21.793 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:21 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/3043673731' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1699237466"}]: dispatch 2024-06-01T02:02:21.793 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:21 smithi161 ceph-mon[26055]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1699237466"}]: dispatch 2024-06-01T02:02:21.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:21 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/4102838325' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/189859058"}]': finished 2024-06-01T02:02:21.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:21 smithi116 ceph-mon[25082]: osdmap e101: 8 total, 8 up, 8 in 2024-06-01T02:02:21.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:21 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/3043673731' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1699237466"}]: dispatch 2024-06-01T02:02:21.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:21 smithi116 ceph-mon[25082]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1699237466"}]: dispatch 2024-06-01T02:02:21.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:21 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/4102838325' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/189859058"}]': finished 2024-06-01T02:02:21.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:21 smithi116 ceph-mon[29958]: osdmap e101: 8 total, 8 up, 8 in 2024-06-01T02:02:21.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:21 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/3043673731' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1699237466"}]: dispatch 2024-06-01T02:02:21.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:21 smithi116 ceph-mon[29958]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1699237466"}]: dispatch 2024-06-01T02:02:22.195 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:21 smithi161 systemd[1]: Stopping Ceph grafana.a for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:02:22.195 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:21 smithi161 bash[55766]: Error: no container with name or ID "ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana.a" found: no such container 2024-06-01T02:02:22.195 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:21 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[44452]: t=2024-06-01T02:02:21+0000 lvl=info msg="Shutdown started" logger=server reason="System signal: terminated" 2024-06-01T02:02:22.195 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:21 smithi161 podman[55778]: 2024-06-01 02:02:21.985666904 +0000 UTC m=+0.090618667 container died f4be12ac83e00bdd54ec4cdcce72efece871406b4199b6e3b657ae2e77721bf6 (image=quay.io/ceph/ceph-grafana:8.3.5, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a, summary=Grafana Container configured for Ceph mgr/dashboard integration, description=Ceph Grafana Container, release=236.1648460182, distribution-scope=public, io.buildah.version=1.24.2, io.openshift.tags=base rhel8, version=8.5, architecture=x86_64, vendor=Red Hat, Inc., maintainer=Paul Cuzner , url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-236.1648460182, name=ubi8, io.openshift.expose-services=, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, build-date=2022-03-28T10:36:18.413762, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.k8s.display-name=Red Hat Universal Base Image 8, vcs-type=git, com.redhat.component=ubi8-container, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com) 2024-06-01T02:02:22.499 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:22 smithi161 podman[55778]: 2024-06-01 02:02:22.366776287 +0000 UTC m=+0.471728077 container cleanup f4be12ac83e00bdd54ec4cdcce72efece871406b4199b6e3b657ae2e77721bf6 (image=quay.io/ceph/ceph-grafana:8.3.5, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.component=ubi8-container, io.openshift.tags=base rhel8, name=ubi8, release=236.1648460182, build-date=2022-03-28T10:36:18.413762, vendor=Red Hat, Inc., io.buildah.version=1.24.2, io.k8s.display-name=Red Hat Universal Base Image 8, maintainer=Paul Cuzner , url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-236.1648460182, distribution-scope=public, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, version=8.5, architecture=x86_64, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, description=Ceph Grafana Container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.expose-services=, vcs-type=git) 2024-06-01T02:02:22.500 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:22 smithi161 bash[55778]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a 2024-06-01T02:02:22.500 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:22 smithi161 bash[55804]: Error: no container with name or ID "ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana.a" found: no such container 2024-06-01T02:02:22.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:22 smithi116 ceph-mon[29958]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-01T02:02:22.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:22 smithi116 ceph-mon[29958]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1699237466"}]': finished 2024-06-01T02:02:22.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:22 smithi116 ceph-mon[29958]: osdmap e102: 8 total, 8 up, 8 in 2024-06-01T02:02:22.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:22 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/2938464096' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1833534794"}]: dispatch 2024-06-01T02:02:22.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:22 smithi116 ceph-mon[29958]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1833534794"}]: dispatch 2024-06-01T02:02:22.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:22 smithi116 ceph-mon[25082]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-01T02:02:22.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:22 smithi116 ceph-mon[25082]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1699237466"}]': finished 2024-06-01T02:02:22.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:22 smithi116 ceph-mon[25082]: osdmap e102: 8 total, 8 up, 8 in 2024-06-01T02:02:22.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:22 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/2938464096' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1833534794"}]: dispatch 2024-06-01T02:02:22.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:22 smithi116 ceph-mon[25082]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1833534794"}]: dispatch 2024-06-01T02:02:22.885 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:22 smithi161 ceph-mon[26055]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-01T02:02:22.886 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:22 smithi161 ceph-mon[26055]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1699237466"}]': finished 2024-06-01T02:02:22.886 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:22 smithi161 ceph-mon[26055]: osdmap e102: 8 total, 8 up, 8 in 2024-06-01T02:02:22.886 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:22 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/2938464096' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1833534794"}]: dispatch 2024-06-01T02:02:22.886 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:22 smithi161 ceph-mon[26055]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1833534794"}]: dispatch 2024-06-01T02:02:22.886 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:22 smithi161 podman[55792]: 2024-06-01 02:02:22.660167791 +0000 UTC m=+0.673091957 container remove f4be12ac83e00bdd54ec4cdcce72efece871406b4199b6e3b657ae2e77721bf6 (image=quay.io/ceph/ceph-grafana:8.3.5, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a, release=236.1648460182, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.expose-services=, vendor=Red Hat, Inc., architecture=x86_64, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.tags=base rhel8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-236.1648460182, build-date=2022-03-28T10:36:18.413762, distribution-scope=public, vcs-type=git, description=Ceph Grafana Container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., version=8.5, com.redhat.component=ubi8-container, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.buildah.version=1.24.2, maintainer=Paul Cuzner , name=ubi8) 2024-06-01T02:02:22.886 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:22 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@grafana.a.service: Deactivated successfully. 2024-06-01T02:02:22.886 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:22 smithi161 systemd[1]: Stopped Ceph grafana.a for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:02:22.886 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:22 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@grafana.a.service: Consumed 2.540s CPU time. 2024-06-01T02:02:22.886 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:22 smithi161 systemd[1]: Starting Ceph grafana.a for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:02:23.176 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:22 smithi161 podman[55866]: 2024-06-01 02:02:22.79166972 +0000 UTC m=+0.016402633 image pull dad864ee21e98e69f4029d1e417aa085001566be0d322fbc75bc6f29b0050c01 quay.io/ceph/ceph-grafana:8.3.5 2024-06-01T02:02:23.176 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:22 smithi161 podman[55866]: 2024-06-01 02:02:22.902630824 +0000 UTC m=+0.127363717 container create 00b0a78ee88e33518e46d676807c921bcf931285c1d4da94ee18cd04599e03ff (image=quay.io/ceph/ceph-grafana:8.3.5, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a, maintainer=Paul Cuzner , io.openshift.tags=base rhel8, name=ubi8, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-type=git, architecture=x86_64, vendor=Red Hat, Inc., com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, summary=Grafana Container configured for Ceph mgr/dashboard integration, version=8.5, io.k8s.display-name=Red Hat Universal Base Image 8, description=Ceph Grafana Container, distribution-scope=public, release=236.1648460182, com.redhat.component=ubi8-container, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-236.1648460182, build-date=2022-03-28T10:36:18.413762, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.openshift.expose-services=, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.buildah.version=1.24.2) 2024-06-01T02:02:23.445 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: [01/Jun/2024:02:02:23] ENGINE Bus STOPPING 2024-06-01T02:02:23.445 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: [01/Jun/2024:02:02:23] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-06-01T02:02:23.445 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: [01/Jun/2024:02:02:23] ENGINE Bus STOPPED 2024-06-01T02:02:23.445 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: [01/Jun/2024:02:02:23] ENGINE Bus STARTING 2024-06-01T02:02:23.446 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 podman[55866]: 2024-06-01 02:02:23.180419465 +0000 UTC m=+0.405152354 container init 00b0a78ee88e33518e46d676807c921bcf931285c1d4da94ee18cd04599e03ff (image=quay.io/ceph/ceph-grafana:8.3.5, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a, vcs-type=git, vendor=Red Hat, Inc., release=236.1648460182, com.redhat.component=ubi8-container, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.openshift.expose-services=, distribution-scope=public, description=Ceph Grafana Container, io.buildah.version=1.24.2, maintainer=Paul Cuzner , summary=Grafana Container configured for Ceph mgr/dashboard integration, build-date=2022-03-28T10:36:18.413762, io.openshift.tags=base rhel8, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., architecture=x86_64, version=8.5, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, io.k8s.display-name=Red Hat Universal Base Image 8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-236.1648460182, name=ubi8) 2024-06-01T02:02:23.446 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 podman[55866]: 2024-06-01 02:02:23.183342459 +0000 UTC m=+0.408075369 container start 00b0a78ee88e33518e46d676807c921bcf931285c1d4da94ee18cd04599e03ff (image=quay.io/ceph/ceph-grafana:8.3.5, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a, build-date=2022-03-28T10:36:18.413762, vendor=Red Hat, Inc., version=8.5, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-236.1648460182, summary=Grafana Container configured for Ceph mgr/dashboard integration, description=Ceph Grafana Container, io.openshift.expose-services=, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-type=git, maintainer=Paul Cuzner , architecture=x86_64, io.openshift.tags=base rhel8, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, distribution-scope=public, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, name=ubi8, com.redhat.component=ubi8-container, release=236.1648460182, io.buildah.version=1.24.2) 2024-06-01T02:02:23.446 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[55888]: t=2024-06-01T02:02:23+0000 lvl=info msg="The state of unified alerting is still not defined. The decision will be made during as we run the database migrations" logger=settings 2024-06-01T02:02:23.446 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[55888]: t=2024-06-01T02:02:23+0000 lvl=warn msg="falling back to legacy setting of 'min_interval_seconds'; please use the configuration option in the `unified_alerting` section if Grafana 8 alerts are enabled." logger=settings 2024-06-01T02:02:23.446 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[55888]: t=2024-06-01T02:02:23+0000 lvl=info msg="Config loaded from" logger=settings file=/usr/share/grafana/conf/defaults.ini 2024-06-01T02:02:23.446 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[55888]: t=2024-06-01T02:02:23+0000 lvl=info msg="Config loaded from" logger=settings file=/etc/grafana/grafana.ini 2024-06-01T02:02:23.446 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[55888]: t=2024-06-01T02:02:23+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_DATA=/var/lib/grafana" 2024-06-01T02:02:23.446 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[55888]: t=2024-06-01T02:02:23+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_LOGS=/var/log/grafana" 2024-06-01T02:02:23.446 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[55888]: t=2024-06-01T02:02:23+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-06-01T02:02:23.447 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[55888]: t=2024-06-01T02:02:23+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-06-01T02:02:23.447 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[55888]: t=2024-06-01T02:02:23+0000 lvl=info msg="Path Home" logger=settings path=/usr/share/grafana 2024-06-01T02:02:23.447 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[55888]: t=2024-06-01T02:02:23+0000 lvl=info msg="Path Data" logger=settings path=/var/lib/grafana 2024-06-01T02:02:23.447 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[55888]: t=2024-06-01T02:02:23+0000 lvl=info msg="Path Logs" logger=settings path=/var/log/grafana 2024-06-01T02:02:23.447 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[55888]: t=2024-06-01T02:02:23+0000 lvl=info msg="Path Plugins" logger=settings path=/var/lib/grafana/plugins 2024-06-01T02:02:23.447 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[55888]: t=2024-06-01T02:02:23+0000 lvl=info msg="Path Provisioning" logger=settings path=/etc/grafana/provisioning 2024-06-01T02:02:23.447 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[55888]: t=2024-06-01T02:02:23+0000 lvl=info msg="App mode production" logger=settings 2024-06-01T02:02:23.447 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[55888]: t=2024-06-01T02:02:23+0000 lvl=info msg="Connecting to DB" logger=sqlstore dbtype=sqlite3 2024-06-01T02:02:23.447 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[55888]: t=2024-06-01T02:02:23+0000 lvl=warn msg="SQLite database file has broader permissions than it should" logger=sqlstore path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-06-01T02:02:23.447 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[55888]: t=2024-06-01T02:02:23+0000 lvl=info msg="Starting DB migrations" logger=migrator 2024-06-01T02:02:23.447 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[55888]: t=2024-06-01T02:02:23+0000 lvl=info msg="migrations completed" logger=migrator performed=0 skipped=377 duration=552.884µs 2024-06-01T02:02:23.447 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[55888]: t=2024-06-01T02:02:23+0000 lvl=info msg="Created default organization" logger=sqlstore 2024-06-01T02:02:23.447 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[55888]: t=2024-06-01T02:02:23+0000 lvl=info msg="Initialising plugins" logger=plugin.manager 2024-06-01T02:02:23.447 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[55888]: t=2024-06-01T02:02:23+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=input 2024-06-01T02:02:23.447 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 bash[55866]: 00b0a78ee88e33518e46d676807c921bcf931285c1d4da94ee18cd04599e03ff 2024-06-01T02:02:23.447 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 systemd[1]: Started Ceph grafana.a for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:02:23.448 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[55888]: t=2024-06-01T02:02:23+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=grafana-piechart-panel 2024-06-01T02:02:23.448 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[55888]: t=2024-06-01T02:02:23+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=vonage-status-panel 2024-06-01T02:02:23.448 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[55888]: t=2024-06-01T02:02:23+0000 lvl=info msg="Live Push Gateway initialization" logger=live.push_http 2024-06-01T02:02:23.448 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[55888]: t=2024-06-01T02:02:23+0000 lvl=info msg="deleted datasource based on configuration" logger=provisioning.datasources name=Dashboard1 2024-06-01T02:02:23.448 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[55888]: t=2024-06-01T02:02:23+0000 lvl=info msg="inserting datasource from configuration " logger=provisioning.datasources name=Dashboard1 uid=P43CA22E17D0F9596 2024-06-01T02:02:23.448 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[55888]: t=2024-06-01T02:02:23+0000 lvl=info msg="inserting datasource from configuration " logger=provisioning.datasources name=Loki uid=P8E80F9AEF21F6940 2024-06-01T02:02:23.448 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[55888]: t=2024-06-01T02:02:23+0000 lvl=info msg="warming cache for startup" logger=ngalert 2024-06-01T02:02:23.448 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[55888]: t=2024-06-01T02:02:23+0000 lvl=info msg="starting MultiOrg Alertmanager" logger=ngalert.multiorg.alertmanager 2024-06-01T02:02:23.448 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[55888]: t=2024-06-01T02:02:23+0000 lvl=info msg="HTTP Server Listen" logger=http.server address=[::]:3000 protocol=https subUrl= socket= 2024-06-01T02:02:23.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:23 smithi116 ceph-mon[25082]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1833534794"}]': finished 2024-06-01T02:02:23.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:23 smithi116 ceph-mon[25082]: osdmap e103: 8 total, 8 up, 8 in 2024-06-01T02:02:23.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:23 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/3940112417' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3243331104"}]: dispatch 2024-06-01T02:02:23.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:23 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:23.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:23 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:23.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:23 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-01T02:02:23.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:23 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi116.front.sepia.ceph.com:9093"}]: dispatch 2024-06-01T02:02:23.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:23 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:23.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:23 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-01T02:02:23.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:23 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi161.front.sepia.ceph.com:9095"}]: dispatch 2024-06-01T02:02:23.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:23 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:23.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:23 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-01T02:02:23.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:23 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi161.front.sepia.ceph.com:3000"}]: dispatch 2024-06-01T02:02:23.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:23 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:23.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:23 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:02:23.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:23 smithi116 ceph-mon[29958]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1833534794"}]': finished 2024-06-01T02:02:23.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:23 smithi116 ceph-mon[29958]: osdmap e103: 8 total, 8 up, 8 in 2024-06-01T02:02:23.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:23 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/3940112417' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3243331104"}]: dispatch 2024-06-01T02:02:23.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:23 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:23.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:23 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:23.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:23 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-01T02:02:23.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:23 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi116.front.sepia.ceph.com:9093"}]: dispatch 2024-06-01T02:02:23.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:23 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:23.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:23 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-01T02:02:23.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:23 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi161.front.sepia.ceph.com:9095"}]: dispatch 2024-06-01T02:02:23.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:23 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:23.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:23 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-01T02:02:23.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:23 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi161.front.sepia.ceph.com:3000"}]: dispatch 2024-06-01T02:02:23.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:23 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:23.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:23 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:02:23.945 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:23.504Z caller=manager.go:609 level=warn component="rule manager" group=nodes msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on(instance) group_left(nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi161\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi161\", job=\"node\", machine=\"x86_64\", nodename=\"smithi161\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", domainname=\"(none)\", instance=\"smithi161\", job=\"node\", machine=\"x86_64\", nodename=\"smithi161\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:02:23.945 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: [01/Jun/2024:02:02:23] ENGINE Serving on http://:::9283 2024-06-01T02:02:23.945 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: [01/Jun/2024:02:02:23] ENGINE Bus STARTED 2024-06-01T02:02:23.945 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: [01/Jun/2024:02:02:23] ENGINE Bus STOPPING 2024-06-01T02:02:23.945 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: [01/Jun/2024:02:02:23] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-06-01T02:02:23.945 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: [01/Jun/2024:02:02:23] ENGINE Bus STOPPED 2024-06-01T02:02:23.945 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: [01/Jun/2024:02:02:23] ENGINE Bus STARTING 2024-06-01T02:02:23.945 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: [01/Jun/2024:02:02:23] ENGINE Serving on http://:::9283 2024-06-01T02:02:23.946 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: [01/Jun/2024:02:02:23] ENGINE Bus STARTED 2024-06-01T02:02:23.946 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: [01/Jun/2024:02:02:23] ENGINE Bus STOPPING 2024-06-01T02:02:23.946 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: [01/Jun/2024:02:02:23] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-06-01T02:02:23.946 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: [01/Jun/2024:02:02:23] ENGINE Bus STOPPED 2024-06-01T02:02:23.946 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: [01/Jun/2024:02:02:23] ENGINE Bus STARTING 2024-06-01T02:02:23.946 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: [01/Jun/2024:02:02:23] ENGINE Serving on http://:::9283 2024-06-01T02:02:23.946 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: [01/Jun/2024:02:02:23] ENGINE Bus STARTED 2024-06-01T02:02:23.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-mon[26055]: from='client.? ' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1833534794"}]': finished 2024-06-01T02:02:23.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-mon[26055]: osdmap e103: 8 total, 8 up, 8 in 2024-06-01T02:02:23.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/3940112417' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3243331104"}]: dispatch 2024-06-01T02:02:23.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:23.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:23.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-01T02:02:23.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi116.front.sepia.ceph.com:9093"}]: dispatch 2024-06-01T02:02:23.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:23.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-01T02:02:23.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi161.front.sepia.ceph.com:9095"}]: dispatch 2024-06-01T02:02:23.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:23.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-01T02:02:23.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi161.front.sepia.ceph.com:3000"}]: dispatch 2024-06-01T02:02:23.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:23.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:23 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:02:24.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:24 smithi161 ceph-mon[26055]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-01T02:02:24.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:24 smithi161 ceph-mon[26055]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-01T02:02:24.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:24 smithi161 ceph-mon[26055]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi116.front.sepia.ceph.com:9093"}]: dispatch 2024-06-01T02:02:24.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:24 smithi161 ceph-mon[26055]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-01T02:02:24.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:24 smithi161 ceph-mon[26055]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi161.front.sepia.ceph.com:9095"}]: dispatch 2024-06-01T02:02:24.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:24 smithi161 ceph-mon[26055]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-01T02:02:24.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:24 smithi161 ceph-mon[26055]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi161.front.sepia.ceph.com:3000"}]: dispatch 2024-06-01T02:02:24.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:24 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/3940112417' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3243331104"}]': finished 2024-06-01T02:02:24.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:24 smithi161 ceph-mon[26055]: osdmap e104: 8 total, 8 up, 8 in 2024-06-01T02:02:24.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:24 smithi116 ceph-mon[29958]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-01T02:02:24.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:24 smithi116 ceph-mon[29958]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-01T02:02:24.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:24 smithi116 ceph-mon[29958]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi116.front.sepia.ceph.com:9093"}]: dispatch 2024-06-01T02:02:24.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:24 smithi116 ceph-mon[29958]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-01T02:02:24.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:24 smithi116 ceph-mon[29958]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi161.front.sepia.ceph.com:9095"}]: dispatch 2024-06-01T02:02:24.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:24 smithi116 ceph-mon[29958]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-01T02:02:24.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:24 smithi116 ceph-mon[29958]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi161.front.sepia.ceph.com:3000"}]: dispatch 2024-06-01T02:02:24.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:24 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/3940112417' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3243331104"}]': finished 2024-06-01T02:02:24.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:24 smithi116 ceph-mon[29958]: osdmap e104: 8 total, 8 up, 8 in 2024-06-01T02:02:24.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:24 smithi116 ceph-mon[25082]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-01T02:02:24.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:24 smithi116 ceph-mon[25082]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-01T02:02:24.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:24 smithi116 ceph-mon[25082]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi116.front.sepia.ceph.com:9093"}]: dispatch 2024-06-01T02:02:24.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:24 smithi116 ceph-mon[25082]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-01T02:02:24.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:24 smithi116 ceph-mon[25082]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi161.front.sepia.ceph.com:9095"}]: dispatch 2024-06-01T02:02:24.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:24 smithi116 ceph-mon[25082]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-01T02:02:24.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:24 smithi116 ceph-mon[25082]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi161.front.sepia.ceph.com:3000"}]: dispatch 2024-06-01T02:02:24.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:24 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/3940112417' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3243331104"}]': finished 2024-06-01T02:02:24.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:24 smithi116 ceph-mon[25082]: osdmap e104: 8 total, 8 up, 8 in 2024-06-01T02:02:26.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:26 smithi116 ceph-mon[25082]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:02:26.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:26 smithi116 ceph-mon[25082]: Upgrade: Updating mgr.y 2024-06-01T02:02:26.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:26 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:26.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:26 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:02:26.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:26 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:02:26.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:26 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-01T02:02:26.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:26 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:02:26.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:26 smithi116 ceph-mon[25082]: Deploying daemon mgr.y on smithi116 2024-06-01T02:02:26.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:26 smithi116 ceph-mon[29958]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:02:26.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:26 smithi116 ceph-mon[29958]: Upgrade: Updating mgr.y 2024-06-01T02:02:26.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:26 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:26.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:26 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:02:26.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:26 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:02:26.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:26 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-01T02:02:26.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:26 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:02:26.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:26 smithi116 ceph-mon[29958]: Deploying daemon mgr.y on smithi116 2024-06-01T02:02:26.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:26 smithi161 ceph-mon[26055]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:02:26.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:26 smithi161 ceph-mon[26055]: Upgrade: Updating mgr.y 2024-06-01T02:02:26.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:26 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:26.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:26 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:02:26.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:26 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:02:26.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:26 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-01T02:02:26.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:26 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:02:26.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:26 smithi161 ceph-mon[26055]: Deploying daemon mgr.y on smithi116 2024-06-01T02:02:27.127 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:26 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:02:26] "GET /metrics HTTP/1.1" 200 38162 "" "Prometheus/2.33.4" 2024-06-01T02:02:27.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:27 smithi116 ceph-mon[25082]: from='client.25159 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:02:27.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:27 smithi116 ceph-mon[29958]: from='client.25159 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:02:27.294 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:27 smithi116 systemd[1]: Stopping Ceph mgr.y for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:02:27.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:27 smithi161 ceph-mon[26055]: from='client.25159 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:02:27.636 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:27 smithi116 podman[65900]: 2024-06-01 02:02:27.328015166 +0000 UTC m=+0.133893455 container died 3c1beb78ef72d2fbfda7f39828beb3f0f8e0a99a40959351b7d023d5bb717fba (image=quay.io/ceph/ceph:v17.2.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y, distribution-scope=public, build-date=2022-05-03T08:36:31.336870, io.k8s.display-name=CentOS Stream 8, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, name=centos-stream, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, io.openshift.expose-services=, vcs-type=git, architecture=x86_64, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, io.openshift.tags=base centos centos-stream, RELEASE=HEAD, GIT_BRANCH=HEAD, vendor=Red Hat, Inc., com.redhat.component=centos-stream-container, version=8, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, GIT_CLEAN=True, CEPH_POINT_RELEASE=-17.2.0, io.buildah.version=1.19.8, release=754, GIT_REPO=https://github.com/ceph/ceph-container.git, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com) 2024-06-01T02:02:28.043 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:27 smithi116 podman[65900]: 2024-06-01 02:02:27.839053156 +0000 UTC m=+0.644931438 container cleanup 3c1beb78ef72d2fbfda7f39828beb3f0f8e0a99a40959351b7d023d5bb717fba (image=quay.io/ceph/ceph:v17.2.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y, io.buildah.version=1.19.8, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, CEPH_POINT_RELEASE=-17.2.0, distribution-scope=public, io.openshift.expose-services=, name=centos-stream, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, io.k8s.display-name=CentOS Stream 8, version=8, release=754, ceph=True, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, vendor=Red Hat, Inc., RELEASE=HEAD, io.openshift.tags=base centos centos-stream, vcs-type=git, build-date=2022-05-03T08:36:31.336870, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., GIT_CLEAN=True, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, com.redhat.component=centos-stream-container, architecture=x86_64, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, com.redhat.license_terms=https://centos.org/legal/licensing-policy/) 2024-06-01T02:02:28.043 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:27 smithi116 bash[65900]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y 2024-06-01T02:02:28.309 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:28 smithi116 ceph-mon[25082]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:02:28.309 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:28 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:02:28.310 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:28 smithi116 ceph-mon[29958]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:02:28.310 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:28 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:02:28.310 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:28 smithi116 podman[65912]: 2024-06-01 02:02:28.048858614 +0000 UTC m=+0.721991695 container remove 3c1beb78ef72d2fbfda7f39828beb3f0f8e0a99a40959351b7d023d5bb717fba (image=quay.io/ceph/ceph:v17.2.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y, vendor=Red Hat, Inc., version=8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, io.openshift.expose-services=, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-17.2.0, architecture=x86_64, vcs-type=git, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, io.k8s.display-name=CentOS Stream 8, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, distribution-scope=public, GIT_CLEAN=True, io.buildah.version=1.19.8, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, RELEASE=HEAD, name=centos-stream, release=754, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, com.redhat.component=centos-stream-container, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., build-date=2022-05-03T08:36:31.336870, io.openshift.tags=base centos centos-stream, ceph=True, GIT_BRANCH=HEAD) 2024-06-01T02:02:28.310 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:28 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-06-01T02:02:28.310 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:28 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-06-01T02:02:28.310 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:28 smithi116 systemd[1]: Stopped Ceph mgr.y for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:02:28.310 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:28 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mgr.y.service: Consumed 1min 2.633s CPU time. 2024-06-01T02:02:28.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:28 smithi161 ceph-mon[26055]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:02:28.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:28 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:02:28.793 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:28 smithi116 systemd[1]: Starting Ceph mgr.y for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:02:28.945 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:28 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:28.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"172.21.15.161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"smithi161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:02:29.700 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:29 smithi116 podman[66178]: 2024-06-01 02:02:29.469292595 +0000 UTC m=+0.850685803 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:02:29.700 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:29 smithi116 podman[66178]: 2024-06-01 02:02:29.589224471 +0000 UTC m=+0.970617669 container create 0af4df0af300650d2dfc40f7cbc8869d89a260b43cb9c6a325dd0032b417b9a0 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-06-01T02:02:29.708 INFO:teuthology.orchestra.run.smithi116.stdout:true 2024-06-01T02:02:30.371 INFO:teuthology.orchestra.run.smithi116.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-01T02:02:30.371 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi116 smithi116 starting - - - - 2024-06-01T02:02:30.371 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi161 smithi161 starting - - - - 2024-06-01T02:02:30.372 INFO:teuthology.orchestra.run.smithi116.stdout:alertmanager.a smithi116 *:9093,9094 starting - - - - 2024-06-01T02:02:30.372 INFO:teuthology.orchestra.run.smithi116.stdout:grafana.a smithi161 *:3000 starting - - - - 2024-06-01T02:02:30.372 INFO:teuthology.orchestra.run.smithi116.stdout:iscsi.foo.smithi116.gjuxzb smithi116 running (4m) 67s ago 4m 48.5M - 3.5 e1d6a67b021e 5b3b199ca2db 2024-06-01T02:02:30.372 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.x smithi161 *:8443,9283 starting - - - - 2024-06-01T02:02:30.372 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.y smithi116 *:9283 running (11m) 67s ago 11m 463M - 17.2.0 e1d6a67b021e 3c1beb78ef72 2024-06-01T02:02:30.372 INFO:teuthology.orchestra.run.smithi116.stdout:mon.a smithi116 running (12m) 67s ago 12m 51.7M 2048M 17.2.0 e1d6a67b021e c821092a538a 2024-06-01T02:02:30.372 INFO:teuthology.orchestra.run.smithi116.stdout:mon.b smithi161 running (10m) 54s ago 10m 38.9M 2048M 17.2.0 e1d6a67b021e 9011986aa6e3 2024-06-01T02:02:30.372 INFO:teuthology.orchestra.run.smithi116.stdout:mon.c smithi116 running (10m) 67s ago 10m 44.1M 2048M 17.2.0 e1d6a67b021e 4a6fe0b16d63 2024-06-01T02:02:30.372 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.a smithi116 *:9100 starting - - - - 2024-06-01T02:02:30.372 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.b smithi161 *:9100 starting - - - - 2024-06-01T02:02:30.372 INFO:teuthology.orchestra.run.smithi116.stdout:osd.0 smithi116 running (10m) 67s ago 9m 53.1M 4096M 17.2.0 e1d6a67b021e 0cc929b0ecad 2024-06-01T02:02:30.372 INFO:teuthology.orchestra.run.smithi116.stdout:osd.1 smithi116 running (9m) 67s ago 9m 54.2M 4096M 17.2.0 e1d6a67b021e 6ff7fc5abe58 2024-06-01T02:02:30.372 INFO:teuthology.orchestra.run.smithi116.stdout:osd.2 smithi116 running (9m) 67s ago 9m 49.9M 4096M 17.2.0 e1d6a67b021e 6aecbb10256b 2024-06-01T02:02:30.372 INFO:teuthology.orchestra.run.smithi116.stdout:osd.3 smithi116 running (8m) 67s ago 8m 52.9M 4096M 17.2.0 e1d6a67b021e a802290f5290 2024-06-01T02:02:30.373 INFO:teuthology.orchestra.run.smithi116.stdout:osd.4 smithi161 running (8m) 54s ago 8m 55.2M 4096M 17.2.0 e1d6a67b021e 70b3f89b0bd9 2024-06-01T02:02:30.373 INFO:teuthology.orchestra.run.smithi116.stdout:osd.5 smithi161 running (8m) 54s ago 8m 49.6M 4096M 17.2.0 e1d6a67b021e 68358b78a25d 2024-06-01T02:02:30.373 INFO:teuthology.orchestra.run.smithi116.stdout:osd.6 smithi161 running (7m) 54s ago 7m 48.4M 4096M 17.2.0 e1d6a67b021e 50abeb0551e7 2024-06-01T02:02:30.373 INFO:teuthology.orchestra.run.smithi116.stdout:osd.7 smithi161 running (7m) 54s ago 7m 52.7M 4096M 17.2.0 e1d6a67b021e 6004e6983735 2024-06-01T02:02:30.373 INFO:teuthology.orchestra.run.smithi116.stdout:prometheus.a smithi161 *:9095 starting - - - - 2024-06-01T02:02:30.373 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi116.vzjrxj smithi116 *:8000 running (4m) 67s ago 4m 72.5M - 17.2.0 e1d6a67b021e df4fe3c1dbc3 2024-06-01T02:02:30.373 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi161.ashqxz smithi161 *:8000 running (4m) 54s ago 4m 73.7M - 17.2.0 e1d6a67b021e 3eecc1f3e2d3 2024-06-01T02:02:30.373 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi116.hokbmf smithi116 *:80 running (4m) 67s ago 4m 73.4M - 17.2.0 e1d6a67b021e 10950ed75dbd 2024-06-01T02:02:30.373 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi161.axjlzw smithi161 *:80 running (4m) 54s ago 4m 72.5M - 17.2.0 e1d6a67b021e 0b5d5ce47772 2024-06-01T02:02:30.495 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:30 smithi161 ceph-mon[26055]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 727 B/s rd, 0 op/s 2024-06-01T02:02:30.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:30 smithi116 ceph-mon[29958]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 727 B/s rd, 0 op/s 2024-06-01T02:02:30.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:30 smithi116 ceph-mon[25082]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 727 B/s rd, 0 op/s 2024-06-01T02:02:30.544 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:30 smithi116 podman[66178]: 2024-06-01 02:02:30.260165483 +0000 UTC m=+1.641558692 container init 0af4df0af300650d2dfc40f7cbc8869d89a260b43cb9c6a325dd0032b417b9a0 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-01T02:02:30.544 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:30 smithi116 podman[66178]: 2024-06-01 02:02:30.263910802 +0000 UTC m=+1.645304003 container start 0af4df0af300650d2dfc40f7cbc8869d89a260b43cb9c6a325dd0032b417b9a0 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-06-01T02:02:30.544 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:30 smithi116 ceph-mgr[66344]: -- 172.21.15.116:0/1637904960 <== mon.0 v2:172.21.15.116:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f849e9c5a0 con 0x55f849e70c00 2024-06-01T02:02:30.544 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:30 smithi116 bash[66178]: 0af4df0af300650d2dfc40f7cbc8869d89a260b43cb9c6a325dd0032b417b9a0 2024-06-01T02:02:30.544 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:30 smithi116 systemd[1]: Started Ceph mgr.y for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:02:30.544 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:30 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:02:30.429+0000 7ff265de2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-01T02:02:30.804 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:02:30.805 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": { 2024-06-01T02:02:30.805 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 3 2024-06-01T02:02:30.805 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:02:30.805 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": { 2024-06-01T02:02:30.805 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 1, 2024-06-01T02:02:30.806 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 1 2024-06-01T02:02:30.806 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:02:30.806 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": { 2024-06-01T02:02:30.806 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2024-06-01T02:02:30.806 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:02:30.806 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": {}, 2024-06-01T02:02:30.806 INFO:teuthology.orchestra.run.smithi116.stdout: "rgw": { 2024-06-01T02:02:30.806 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-06-01T02:02:30.806 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:02:30.806 INFO:teuthology.orchestra.run.smithi116.stdout: "overall": { 2024-06-01T02:02:30.806 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 16, 2024-06-01T02:02:30.807 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 1 2024-06-01T02:02:30.807 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T02:02:30.807 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:02:31.043 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:30 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:02:30.636+0000 7ff265de2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-01T02:02:31.043 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:30 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:02:30.743+0000 7ff265de2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-01T02:02:31.152 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:02:31.152 INFO:teuthology.orchestra.run.smithi116.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-01T02:02:31.152 INFO:teuthology.orchestra.run.smithi116.stdout: "in_progress": true, 2024-06-01T02:02:31.152 INFO:teuthology.orchestra.run.smithi116.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-01T02:02:31.153 INFO:teuthology.orchestra.run.smithi116.stdout: "services_complete": [], 2024-06-01T02:02:31.153 INFO:teuthology.orchestra.run.smithi116.stdout: "progress": "", 2024-06-01T02:02:31.153 INFO:teuthology.orchestra.run.smithi116.stdout: "message": "Currently upgrading mgr daemons", 2024-06-01T02:02:31.153 INFO:teuthology.orchestra.run.smithi116.stdout: "is_paused": false 2024-06-01T02:02:31.153 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:02:31.293 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:31 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:02:31.076+0000 7ff265de2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-01T02:02:31.294 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:31 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:02:31.169+0000 7ff265de2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-01T02:02:31.580 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:31 smithi116 ceph-mon[25082]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:02:31.580 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:31 smithi116 ceph-mon[25082]: from='client.25219 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:02:31.580 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:31 smithi116 ceph-mon[25082]: from='client.25225 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:02:31.580 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:31 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:31.580 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:31 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:31.580 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:31 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:02:31.580 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:31 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:02:31.580 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:31 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:02:31.580 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:31 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:02:31.580 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:31 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/2464048100' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:02:31.581 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:31 smithi116 ceph-mon[29958]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:02:31.581 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:31 smithi116 ceph-mon[29958]: from='client.25219 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:02:31.581 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:31 smithi116 ceph-mon[29958]: from='client.25225 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:02:31.581 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:31 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:31.581 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:31 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:31.581 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:31 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:02:31.581 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:31 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:02:31.581 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:31 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:02:31.581 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:31 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:02:31.581 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:31 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/2464048100' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:02:31.582 INFO:teuthology.orchestra.run.smithi116.stdout:HEALTH_OK 2024-06-01T02:02:31.690 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:31 smithi161 ceph-mon[26055]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:02:31.690 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:31 smithi161 ceph-mon[26055]: from='client.25219 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:02:31.690 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:31 smithi161 ceph-mon[26055]: from='client.25225 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:02:31.691 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:31 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:31.691 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:31 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:31.691 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:31 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:02:31.691 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:31 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:02:31.691 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:31 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:02:31.691 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:31 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:02:31.691 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:31 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/2464048100' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:02:32.512 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:32 smithi116 ceph-mon[25082]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:02:32.513 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:32 smithi116 ceph-mon[25082]: Deploying daemon agent.smithi116 on smithi116 2024-06-01T02:02:32.513 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:32 smithi116 ceph-mon[25082]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:02:32.513 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:32 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1524889405' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:02:32.513 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:32 smithi116 ceph-mon[29958]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:02:32.513 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:32 smithi116 ceph-mon[29958]: Deploying daemon agent.smithi116 on smithi116 2024-06-01T02:02:32.513 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:32 smithi116 ceph-mon[29958]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:02:32.513 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:32 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1524889405' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:02:32.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:32 smithi161 ceph-mon[26055]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:02:32.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:32 smithi161 ceph-mon[26055]: Deploying daemon agent.smithi116 on smithi116 2024-06-01T02:02:32.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:32 smithi161 ceph-mon[26055]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:02:32.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:32 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1524889405' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:02:32.793 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:32 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:02:32.515+0000 7ff265de2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-01T02:02:32.793 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:32 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:02:32.698+0000 7ff265de2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-01T02:02:33.196 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:32 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:02:32.875+0000 7ff265de2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-01T02:02:33.196 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:33 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:02:33.199+0000 7ff265de2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-01T02:02:33.453 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:33 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:02:33.331+0000 7ff265de2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-01T02:02:33.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:33 smithi161 ceph-mon[26055]: Deploying daemon agent.smithi161 on smithi161 2024-06-01T02:02:33.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:33 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:33.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:33 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:33.587 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:33 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:33.505Z caller=manager.go:609 level=warn component="rule manager" group=nodes msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on(instance) group_left(nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi116\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:02:33.755 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:33 smithi116 ceph-mon[29958]: Deploying daemon agent.smithi161 on smithi161 2024-06-01T02:02:33.755 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:33 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:33.755 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:33 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:33.755 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:33 smithi116 ceph-mon[25082]: Deploying daemon agent.smithi161 on smithi161 2024-06-01T02:02:33.755 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:33 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:33.756 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:33 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:33.756 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:33 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:02:33.550+0000 7ff265de2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-01T02:02:33.756 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:33 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:02:33.758+0000 7ff265de2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-01T02:02:34.633 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:34 smithi161 ceph-mon[26055]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:02:34.633 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:34 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:34.634 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:34 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:02:34.634 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:34 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:02:34.634 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:34 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:34.634 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:34 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi116.gjuxzb", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-01T02:02:34.634 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:34 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi116.gjuxzb", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-01T02:02:34.634 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:34 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:02:34.634 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:34 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:34.709 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:34 smithi116 ceph-mon[25082]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:02:34.709 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:34 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:34.710 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:34 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:02:34.710 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:34 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:02:34.710 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:34 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:34.710 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:34 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi116.gjuxzb", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-01T02:02:34.710 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:34 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi116.gjuxzb", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-01T02:02:34.710 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:34 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:02:34.710 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:34 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:34.710 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:34 smithi116 ceph-mon[29958]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:02:34.710 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:34 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:34.710 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:34 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:02:34.711 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:34 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:02:34.711 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:34 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:34.711 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:34 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi116.gjuxzb", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-01T02:02:34.711 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:34 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi116.gjuxzb", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-01T02:02:34.711 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:34 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:02:34.711 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:34 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:34.711 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:34 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:02:34.454+0000 7ff265de2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-01T02:02:35.013 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:34 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:02:34.916+0000 7ff265de2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-01T02:02:35.289 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:35 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:02:35.016+0000 7ff265de2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-01T02:02:35.289 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:35 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:02:35.238+0000 7ff265de2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-01T02:02:35.526 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:35 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:02:35.526 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:35 smithi161 ceph-mon[26055]: Reconfiguring iscsi.foo.smithi116.gjuxzb (dependencies changed)... 2024-06-01T02:02:35.526 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:35 smithi161 ceph-mon[26055]: Reconfiguring daemon iscsi.foo.smithi116.gjuxzb on smithi116 2024-06-01T02:02:35.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:35 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:02:35.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:35 smithi116 ceph-mon[25082]: Reconfiguring iscsi.foo.smithi116.gjuxzb (dependencies changed)... 2024-06-01T02:02:35.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:35 smithi116 ceph-mon[25082]: Reconfiguring daemon iscsi.foo.smithi116.gjuxzb on smithi116 2024-06-01T02:02:35.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:35 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:02:35.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:35 smithi116 ceph-mon[29958]: Reconfiguring iscsi.foo.smithi116.gjuxzb (dependencies changed)... 2024-06-01T02:02:35.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:35 smithi116 ceph-mon[29958]: Reconfiguring daemon iscsi.foo.smithi116.gjuxzb on smithi116 2024-06-01T02:02:35.876 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:35 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:02:35.778+0000 7ff265de2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-01T02:02:35.876 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:35 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:02:35.878+0000 7ff265de2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-01T02:02:36.463 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:36 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:02:36.133+0000 7ff265de2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-01T02:02:36.463 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:36 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:02:36.276+0000 7ff265de2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-01T02:02:36.463 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:36 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:02:36.376+0000 7ff265de2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-01T02:02:36.588 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:36 smithi161 ceph-mon[26055]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-06-01T02:02:36.588 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:36 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:36.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:36 smithi116 ceph-mon[29958]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-06-01T02:02:36.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:36 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:36.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:36 smithi116 ceph-mon[25082]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-06-01T02:02:36.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:36 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:36.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:36 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:02:36.502+0000 7ff265de2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-01T02:02:36.944 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:36 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:02:36] "GET /metrics HTTP/1.1" 200 38156 "" "Prometheus/2.33.4" 2024-06-01T02:02:37.136 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:36 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:02:36.816+0000 7ff265de2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-01T02:02:37.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:37 smithi161 ceph-mon[26055]: from='client.25159 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:02:37.776 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:37 smithi116 ceph-mon[25082]: from='client.25159 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:02:37.776 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:37 smithi116 ceph-mon[29958]: from='client.25159 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:02:37.776 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:37 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:02:37.442+0000 7ff265de2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-01T02:02:37.776 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:37 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:02:37.670+0000 7ff265de2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-01T02:02:37.776 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:37 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: [01/Jun/2024:02:02:37] ENGINE Bus STARTING 2024-06-01T02:02:37.776 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:37 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: CherryPy Checker: 2024-06-01T02:02:37.777 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:37 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: The Application mounted at '' has an empty config. 2024-06-01T02:02:37.777 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:37 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:02:37.777 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:37 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: [01/Jun/2024:02:02:37] ENGINE Serving on http://:::9283 2024-06-01T02:02:37.777 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:37 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: [01/Jun/2024:02:02:37] ENGINE Bus STARTED 2024-06-01T02:02:38.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:38 smithi116 ceph-mon[25082]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:02:38.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:38 smithi116 ceph-mon[25082]: Standby manager daemon y restarted 2024-06-01T02:02:38.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:38 smithi116 ceph-mon[25082]: Standby manager daemon y started 2024-06-01T02:02:38.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:38 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-01T02:02:38.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:38 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T02:02:38.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:38 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-01T02:02:38.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:38 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T02:02:38.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:38 smithi116 ceph-mon[29958]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:02:38.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:38 smithi116 ceph-mon[29958]: Standby manager daemon y restarted 2024-06-01T02:02:38.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:38 smithi116 ceph-mon[29958]: Standby manager daemon y started 2024-06-01T02:02:38.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:38 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-01T02:02:38.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:38 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T02:02:38.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:38 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-01T02:02:38.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:38 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T02:02:38.758 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:38 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:38.658Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"172.21.15.161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"smithi161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:02:38.758 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:38 smithi161 ceph-mon[26055]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:02:38.758 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:38 smithi161 ceph-mon[26055]: Standby manager daemon y restarted 2024-06-01T02:02:38.758 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:38 smithi161 ceph-mon[26055]: Standby manager daemon y started 2024-06-01T02:02:38.758 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:38 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-01T02:02:38.759 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:38 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T02:02:38.759 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:38 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-01T02:02:38.759 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:38 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T02:02:39.591 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:39 smithi116 ceph-mon[25082]: mgrmap e26: x(active, since 56s), standbys: y 2024-06-01T02:02:39.591 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:39 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:39.591 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:39 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:39.591 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:39 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:39.591 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:39 smithi116 ceph-mon[29958]: mgrmap e26: x(active, since 56s), standbys: y 2024-06-01T02:02:39.591 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:39 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:39.591 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:39 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:39.591 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:39 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:39.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:39 smithi161 ceph-mon[26055]: mgrmap e26: x(active, since 56s), standbys: y 2024-06-01T02:02:39.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:39 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:39.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:39 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:39.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:39 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:40.637 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:40 smithi116 ceph-mon[25082]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:02:40.637 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:40 smithi116 ceph-mon[29958]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:02:40.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:40 smithi161 ceph-mon[26055]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:02:41.687 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:41 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:41.687 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:41 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:41.687 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:41 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-06-01T02:02:41.687 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:41 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-06-01T02:02:41.687 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:41 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:02:41.687 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:41 smithi116 ceph-mon[25082]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-06-01T02:02:41.687 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:41 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-06-01T02:02:41.687 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:41 smithi116 ceph-mon[25082]: osdmap e105: 8 total, 8 up, 8 in 2024-06-01T02:02:41.687 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:41 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/308441192' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-01T02:02:41.688 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:41 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:41.688 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:41 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:41.688 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:41 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-06-01T02:02:41.688 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:41 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-06-01T02:02:41.688 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:41 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:02:41.688 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:41 smithi116 ceph-mon[29958]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-06-01T02:02:41.688 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:41 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-06-01T02:02:41.689 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:41 smithi116 ceph-mon[29958]: osdmap e105: 8 total, 8 up, 8 in 2024-06-01T02:02:41.689 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:41 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/308441192' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-01T02:02:41.689 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:41 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: [01/Jun/2024:02:02:41] ENGINE Bus STOPPING 2024-06-01T02:02:41.689 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:41 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: [01/Jun/2024:02:02:41] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-06-01T02:02:41.689 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:41 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: [01/Jun/2024:02:02:41] ENGINE Bus STOPPED 2024-06-01T02:02:41.689 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:41 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: [01/Jun/2024:02:02:41] ENGINE Bus STARTING 2024-06-01T02:02:41.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:41 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:41.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:41 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' 2024-06-01T02:02:41.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:41 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-06-01T02:02:41.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:41 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-06-01T02:02:41.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:41 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:02:41.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:41 smithi161 ceph-mon[26055]: from='mgr.24875 172.21.15.161:0/4157522808' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-06-01T02:02:41.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:41 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-06-01T02:02:41.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:41 smithi161 ceph-mon[26055]: osdmap e105: 8 total, 8 up, 8 in 2024-06-01T02:02:41.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:41 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/308441192' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-01T02:02:41.946 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:41 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:02:41.545+0000 7fc45f668700 -1 mgr handle_mgr_map I was active but no longer am 2024-06-01T02:02:41.946 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:41 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: ignoring --setuser ceph since I am not root 2024-06-01T02:02:41.946 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:41 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: ignoring --setgroup ceph since I am not root 2024-06-01T02:02:41.946 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:41 smithi161 ceph-mgr[52018]: -- 172.21.15.161:0/1823149557 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5622174de5a0 con 0x5622174b2800 2024-06-01T02:02:41.946 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:41 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:02:41.766+0000 7f44d670f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-01T02:02:42.043 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:41 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: [01/Jun/2024:02:02:41] ENGINE Serving on http://:::9283 2024-06-01T02:02:42.043 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:41 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: [01/Jun/2024:02:02:41] ENGINE Bus STARTED 2024-06-01T02:02:42.404 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:41 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:02:41.988+0000 7f44d670f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-01T02:02:42.405 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:02:42.096+0000 7f44d670f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-01T02:02:42.405 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:02:42.407+0000 7f44d670f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-01T02:02:42.695 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:02:42.494+0000 7f44d670f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-01T02:02:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-mon[26055]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:02:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-mon[26055]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-06-01T02:02:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-mon[26055]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-06-01T02:02:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-mon[26055]: Upgrade: Need to upgrade myself (mgr.x) 2024-06-01T02:02:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-mon[26055]: Upgrade: Need to upgrade myself (mgr.x) 2024-06-01T02:02:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-mon[26055]: Failing over to other MGR 2024-06-01T02:02:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-mon[26055]: from='mgr.24875 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-06-01T02:02:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-mon[26055]: mgrmap e27: y(active, starting, since 0.950369s) 2024-06-01T02:02:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T02:02:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T02:02:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T02:02:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-01T02:02:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T02:02:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T02:02:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T02:02:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T02:02:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T02:02:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T02:02:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T02:02:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T02:02:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-01T02:02:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-01T02:02:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-01T02:02:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-mon[26055]: Manager daemon y is now available 2024-06-01T02:02:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:02:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:02:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:02:42.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T02:02:42.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T02:02:42.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:42 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:02:42.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[25082]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:02:42.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[25082]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-06-01T02:02:42.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[25082]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-06-01T02:02:42.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[25082]: Upgrade: Need to upgrade myself (mgr.x) 2024-06-01T02:02:42.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[25082]: Upgrade: Need to upgrade myself (mgr.x) 2024-06-01T02:02:42.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[25082]: Failing over to other MGR 2024-06-01T02:02:42.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[25082]: from='mgr.24875 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-06-01T02:02:42.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[25082]: mgrmap e27: y(active, starting, since 0.950369s) 2024-06-01T02:02:42.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T02:02:42.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T02:02:42.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T02:02:42.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-01T02:02:42.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T02:02:42.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T02:02:42.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T02:02:42.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T02:02:42.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T02:02:42.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T02:02:42.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T02:02:42.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T02:02:42.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-01T02:02:42.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-01T02:02:42.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-01T02:02:42.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[25082]: Manager daemon y is now available 2024-06-01T02:02:42.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:42.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:02:42.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:02:42.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:02:42.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[29958]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:02:42.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[29958]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-06-01T02:02:42.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[29958]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-06-01T02:02:42.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[29958]: Upgrade: Need to upgrade myself (mgr.x) 2024-06-01T02:02:42.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[29958]: Upgrade: Need to upgrade myself (mgr.x) 2024-06-01T02:02:42.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[29958]: Failing over to other MGR 2024-06-01T02:02:42.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[29958]: from='mgr.24875 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-06-01T02:02:42.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[29958]: mgrmap e27: y(active, starting, since 0.950369s) 2024-06-01T02:02:42.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T02:02:42.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T02:02:42.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T02:02:42.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-01T02:02:42.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T02:02:42.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T02:02:42.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T02:02:42.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T02:02:42.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T02:02:42.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T02:02:42.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T02:02:42.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T02:02:42.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-01T02:02:42.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-01T02:02:42.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-01T02:02:42.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[29958]: Manager daemon y is now available 2024-06-01T02:02:42.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:42.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:02:42.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:02:42.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:02:42.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T02:02:42.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T02:02:42.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:02:42.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-01T02:02:42.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-01T02:02:42.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:42 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:02:43.606 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:43 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:43.506Z caller=manager.go:609 level=warn component="rule manager" group=nodes msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on(instance) group_left(nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi116\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:02:43.606 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:43 smithi161 ceph-mon[26055]: mgrmap e28: y(active, since 1.96069s) 2024-06-01T02:02:43.606 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:43 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi116", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:02:43.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:43 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:02:43.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:43 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:02:43.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:43 smithi161 ceph-mon[26055]: Updating smithi116:/etc/ceph/ceph.conf 2024-06-01T02:02:43.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:43 smithi161 ceph-mon[26055]: Updating smithi161:/etc/ceph/ceph.conf 2024-06-01T02:02:43.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:43 smithi161 ceph-mon[26055]: [01/Jun/2024:02:02:42] ENGINE Bus STARTING 2024-06-01T02:02:43.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:43 smithi161 ceph-mon[26055]: [01/Jun/2024:02:02:42] ENGINE Serving on https://172.21.15.116:7150 2024-06-01T02:02:43.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:43 smithi161 ceph-mon[26055]: [01/Jun/2024:02:02:42] ENGINE Serving on http://172.21.15.116:8765 2024-06-01T02:02:43.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:43 smithi161 ceph-mon[26055]: [01/Jun/2024:02:02:42] ENGINE Bus STARTED 2024-06-01T02:02:43.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:43 smithi161 ceph-mon[26055]: Updating smithi161:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-01T02:02:43.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:43 smithi161 ceph-mon[26055]: Updating smithi116:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-01T02:02:43.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:43 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:43.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:43 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:43.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:43 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:43.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:43 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:43.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:43 smithi161 ceph-mon[26055]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-06-01T02:02:43.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:43 smithi161 ceph-mon[26055]: Updating smithi116:/etc/ceph/ceph.client.admin.keyring 2024-06-01T02:02:43.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:43 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:43.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:43 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:43.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[25082]: mgrmap e28: y(active, since 1.96069s) 2024-06-01T02:02:43.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi116", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:02:43.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:02:43.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:02:43.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[25082]: Updating smithi116:/etc/ceph/ceph.conf 2024-06-01T02:02:43.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[25082]: Updating smithi161:/etc/ceph/ceph.conf 2024-06-01T02:02:43.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[25082]: [01/Jun/2024:02:02:42] ENGINE Bus STARTING 2024-06-01T02:02:43.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[25082]: [01/Jun/2024:02:02:42] ENGINE Serving on https://172.21.15.116:7150 2024-06-01T02:02:43.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[25082]: [01/Jun/2024:02:02:42] ENGINE Serving on http://172.21.15.116:8765 2024-06-01T02:02:43.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[25082]: [01/Jun/2024:02:02:42] ENGINE Bus STARTED 2024-06-01T02:02:43.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[25082]: Updating smithi161:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-01T02:02:43.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[25082]: Updating smithi116:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-01T02:02:43.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:43.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:43.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:43.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:43.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[25082]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-06-01T02:02:43.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[25082]: Updating smithi116:/etc/ceph/ceph.client.admin.keyring 2024-06-01T02:02:43.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:43.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:43.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[29958]: mgrmap e28: y(active, since 1.96069s) 2024-06-01T02:02:43.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi116", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:02:43.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:02:43.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:02:43.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[29958]: Updating smithi116:/etc/ceph/ceph.conf 2024-06-01T02:02:43.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[29958]: Updating smithi161:/etc/ceph/ceph.conf 2024-06-01T02:02:43.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[29958]: [01/Jun/2024:02:02:42] ENGINE Bus STARTING 2024-06-01T02:02:43.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[29958]: [01/Jun/2024:02:02:42] ENGINE Serving on https://172.21.15.116:7150 2024-06-01T02:02:43.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[29958]: [01/Jun/2024:02:02:42] ENGINE Serving on http://172.21.15.116:8765 2024-06-01T02:02:43.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[29958]: [01/Jun/2024:02:02:42] ENGINE Bus STARTED 2024-06-01T02:02:43.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[29958]: Updating smithi161:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-01T02:02:43.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[29958]: Updating smithi116:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-01T02:02:43.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:43.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:43.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:43.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:43.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[29958]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-06-01T02:02:43.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[29958]: Updating smithi116:/etc/ceph/ceph.client.admin.keyring 2024-06-01T02:02:43.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:43.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:43 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:43.944 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:43 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:02:43.798+0000 7f44d670f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-01T02:02:44.244 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:43 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:02:43.975+0000 7f44d670f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-01T02:02:44.244 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:44 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:02:44.154+0000 7f44d670f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-01T02:02:44.574 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:44 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:02:44.471+0000 7f44d670f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-01T02:02:44.924 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:44 smithi116 ceph-mon[25082]: Updating smithi161:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-01T02:02:44.924 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:44 smithi116 ceph-mon[25082]: Updating smithi116:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-01T02:02:44.925 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:44 smithi116 ceph-mon[25082]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:44.925 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:44 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:44.925 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:44 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:44.925 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:44 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:44.925 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:44 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:44.925 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:44 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:44.925 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:44 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:02:44.925 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:44 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:44.925 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:44 smithi116 ceph-mon[25082]: Reconfiguring iscsi.foo.smithi116.gjuxzb (dependencies changed)... 2024-06-01T02:02:44.925 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:44 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi116.gjuxzb", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-01T02:02:44.925 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:44 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:02:44.925 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:44 smithi116 ceph-mon[25082]: Reconfiguring daemon iscsi.foo.smithi116.gjuxzb on smithi116 2024-06-01T02:02:44.926 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:44 smithi116 ceph-mon[25082]: mgrmap e29: y(active, since 3s) 2024-06-01T02:02:44.926 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:44 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:44.926 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:44 smithi116 ceph-mon[29958]: Updating smithi161:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-01T02:02:44.926 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:44 smithi116 ceph-mon[29958]: Updating smithi116:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-01T02:02:44.926 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:44 smithi116 ceph-mon[29958]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:44.926 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:44 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:44.926 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:44 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:44.926 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:44 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:44.926 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:44 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:44.926 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:44 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:44.927 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:44 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:02:44.927 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:44 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:44.927 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:44 smithi116 ceph-mon[29958]: Reconfiguring iscsi.foo.smithi116.gjuxzb (dependencies changed)... 2024-06-01T02:02:44.927 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:44 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi116.gjuxzb", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-01T02:02:44.927 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:44 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:02:44.927 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:44 smithi116 ceph-mon[29958]: Reconfiguring daemon iscsi.foo.smithi116.gjuxzb on smithi116 2024-06-01T02:02:44.927 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:44 smithi116 ceph-mon[29958]: mgrmap e29: y(active, since 3s) 2024-06-01T02:02:44.927 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:44 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:44 smithi161 ceph-mon[26055]: Updating smithi161:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-01T02:02:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:44 smithi161 ceph-mon[26055]: Updating smithi116:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-01T02:02:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:44 smithi161 ceph-mon[26055]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:44 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:44 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:44 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:44 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:44 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:44 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:02:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:44 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:44 smithi161 ceph-mon[26055]: Reconfiguring iscsi.foo.smithi116.gjuxzb (dependencies changed)... 2024-06-01T02:02:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:44 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi116.gjuxzb", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-01T02:02:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:44 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:02:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:44 smithi161 ceph-mon[26055]: Reconfiguring daemon iscsi.foo.smithi116.gjuxzb on smithi116 2024-06-01T02:02:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:44 smithi161 ceph-mon[26055]: mgrmap e29: y(active, since 3s) 2024-06-01T02:02:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:44 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:44.946 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:44 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:02:44.576+0000 7f44d670f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-01T02:02:44.946 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:44 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:02:44.797+0000 7f44d670f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-01T02:02:45.444 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:45 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:02:45.019+0000 7f44d670f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-01T02:02:45.844 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:45 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:02:45.663+0000 7f44d670f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-01T02:02:46.273 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:46 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:46.273 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:46 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:46.273 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:46 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:46.273 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:46 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:46.273 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:46 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:46.273 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:46 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:46.376 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:46 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:46.376 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:46 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:46.376 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:46 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:46.376 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:46 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:02:46.100+0000 7f44d670f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-01T02:02:46.376 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:46 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:02:46.207+0000 7f44d670f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-01T02:02:46.656 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:46 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:02:46.437+0000 7f44d670f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-01T02:02:47.338 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:46 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:02:46.993+0000 7f44d670f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-01T02:02:47.338 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:47 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:02:47.090+0000 7f44d670f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-01T02:02:47.338 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:47 smithi161 ceph-mon[26055]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:47.338 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:47 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:47.338 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:47 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:47.339 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:47 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:47.339 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:47 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:47.339 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:47 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:47.339 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:47 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:47.435 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-mon[29958]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:47.435 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:47.435 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:47.435 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:47.435 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:47.436 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:47.436 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:47.436 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-mon[25082]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:47.436 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:47.436 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:47.436 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:47.436 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:47.436 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:47.436 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:47.694 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:47 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:02:47.340+0000 7f44d670f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-01T02:02:47.695 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:47 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:02:47.480+0000 7f44d670f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-01T02:02:47.695 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:47 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:02:47.588+0000 7f44d670f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-01T02:02:47.997 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:47 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:02:47.714+0000 7f44d670f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-01T02:02:48.011 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:02:47.738+0000 7ff1eb968700 -1 log_channel(cephadm) log [ERR] : cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi116.gjuxzb ... 2024-06-01T02:02:48.011 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: Creating ceph-iscsi config... 2024-06-01T02:02:48.011 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/iscsi-gateway.cfg 2024-06-01T02:02:48.012 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/tcmu-runner-entrypoint.sh 2024-06-01T02:02:48.012 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: Non-zero exit code 1 from systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb 2024-06-01T02:02:48.012 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: systemctl: stderr Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:02:48.012 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: systemctl: stderr See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:02:48.012 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: Traceback (most recent call last): 2024-06-01T02:02:48.012 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-06-01T02:02:48.013 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: return _run_code(code, main_globals, None, 2024-06-01T02:02:48.013 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-06-01T02:02:48.013 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: exec(code, run_globals) 2024-06-01T02:02:48.013 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5618, in 2024-06-01T02:02:48.013 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5606, in main 2024-06-01T02:02:48.013 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3096, in command_deploy_from 2024-06-01T02:02:48.013 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3131, in _common_deploy 2024-06-01T02:02:48.013 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3151, in _deploy_daemon_container 2024-06-01T02:02:48.014 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 1123, in deploy_daemon 2024-06-01T02:02:48.014 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-06-01T02:02:48.014 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: RuntimeError: Failed command: systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb: Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:02:48.014 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:02:48.015 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: Traceback (most recent call last): 2024-06-01T02:02:48.015 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1114, in _check_daemons 2024-06-01T02:02:48.015 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: self.mgr._daemon_action(daemon_spec, action=action) 2024-06-01T02:02:48.015 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/usr/share/ceph/mgr/cephadm/module.py", line 2457, in _daemon_action 2024-06-01T02:02:48.015 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: CephadmServe(self)._create_daemon(daemon_spec, reconfig=(action == 'reconfig'))) 2024-06-01T02:02:48.015 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/usr/share/ceph/mgr/cephadm/module.py", line 779, in wait_async 2024-06-01T02:02:48.015 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: return self.event_loop.get_result(coro, timeout) 2024-06-01T02:02:48.015 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/usr/share/ceph/mgr/cephadm/ssh.py", line 136, in get_result 2024-06-01T02:02:48.015 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: return future.result(timeout) 2024-06-01T02:02:48.015 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/lib64/python3.6/concurrent/futures/_base.py", line 432, in result 2024-06-01T02:02:48.015 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: return self.__get_result() 2024-06-01T02:02:48.015 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result 2024-06-01T02:02:48.015 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: raise self._exception 2024-06-01T02:02:48.015 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1377, in _create_daemon 2024-06-01T02:02:48.016 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: use_current_daemon_image=reconfig, 2024-06-01T02:02:48.016 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1692, in _run_cephadm 2024-06-01T02:02:48.016 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: f'cephadm exited with an error code: {code}, stderr: {err}') 2024-06-01T02:02:48.016 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi116.gjuxzb ... 2024-06-01T02:02:48.016 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: Creating ceph-iscsi config... 2024-06-01T02:02:48.016 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/iscsi-gateway.cfg 2024-06-01T02:02:48.016 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/tcmu-runner-entrypoint.sh 2024-06-01T02:02:48.016 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: Non-zero exit code 1 from systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb 2024-06-01T02:02:48.016 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: systemctl: stderr Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:02:48.016 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: systemctl: stderr See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:02:48.016 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: Traceback (most recent call last): 2024-06-01T02:02:48.016 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-06-01T02:02:48.017 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: return _run_code(code, main_globals, None, 2024-06-01T02:02:48.017 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-06-01T02:02:48.017 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: exec(code, run_globals) 2024-06-01T02:02:48.017 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5618, in 2024-06-01T02:02:48.017 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5606, in main 2024-06-01T02:02:48.017 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3096, in command_deploy_from 2024-06-01T02:02:48.017 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3131, in _common_deploy 2024-06-01T02:02:48.017 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3151, in _deploy_daemon_container 2024-06-01T02:02:48.017 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 1123, in deploy_daemon 2024-06-01T02:02:48.017 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-06-01T02:02:48.017 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: RuntimeError: Failed command: systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb: Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:02:48.017 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:02:48.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:48 smithi116 ceph-mon[29958]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:48.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:48 smithi116 ceph-mon[29958]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-06-01T02:02:48.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:48 smithi116 ceph-mon[25082]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:48.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:48 smithi116 ceph-mon[25082]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-06-01T02:02:48.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:48 smithi161 ceph-mon[26055]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:48.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:48 smithi161 ceph-mon[26055]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-06-01T02:02:48.446 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:48 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:02:48.029+0000 7f44d670f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-01T02:02:48.944 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:48 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:02:48.687+0000 7f44d670f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-01T02:02:48.944 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:48 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:02:48.919+0000 7f44d670f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-01T02:02:48.945 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:48 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: [01/Jun/2024:02:02:48] ENGINE Bus STARTING 2024-06-01T02:02:49.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:49 smithi161 ceph-mon[26055]: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi116.gjuxzb ... 2024-06-01T02:02:49.361 INFO:journalctl@ceph.mon.b.smithi161.stdout: Creating ceph-iscsi config... 2024-06-01T02:02:49.361 INFO:journalctl@ceph.mon.b.smithi161.stdout: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/iscsi-gateway.cfg 2024-06-01T02:02:49.361 INFO:journalctl@ceph.mon.b.smithi161.stdout: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/tcmu-runner-entrypoint.sh 2024-06-01T02:02:49.361 INFO:journalctl@ceph.mon.b.smithi161.stdout: Non-zero exit code 1 from systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb 2024-06-01T02:02:49.361 INFO:journalctl@ceph.mon.b.smithi161.stdout: systemctl: stderr Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:02:49.361 INFO:journalctl@ceph.mon.b.smithi161.stdout: systemctl: stderr See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:02:49.361 INFO:journalctl@ceph.mon.b.smithi161.stdout: Traceback (most recent call last): 2024-06-01T02:02:49.361 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-06-01T02:02:49.361 INFO:journalctl@ceph.mon.b.smithi161.stdout: return _run_code(code, main_globals, None, 2024-06-01T02:02:49.361 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-06-01T02:02:49.362 INFO:journalctl@ceph.mon.b.smithi161.stdout: exec(code, run_globals) 2024-06-01T02:02:49.362 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5618, in 2024-06-01T02:02:49.362 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5606, in main 2024-06-01T02:02:49.362 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3096, in command_deploy_from 2024-06-01T02:02:49.362 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3131, in _common_deploy 2024-06-01T02:02:49.362 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3151, in _deploy_daemon_container 2024-06-01T02:02:49.362 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 1123, in deploy_daemon 2024-06-01T02:02:49.362 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-06-01T02:02:49.362 INFO:journalctl@ceph.mon.b.smithi161.stdout: RuntimeError: Failed command: systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb: Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:02:49.362 INFO:journalctl@ceph.mon.b.smithi161.stdout: See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:02:49.362 INFO:journalctl@ceph.mon.b.smithi161.stdout: Traceback (most recent call last): 2024-06-01T02:02:49.362 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1114, in _check_daemons 2024-06-01T02:02:49.363 INFO:journalctl@ceph.mon.b.smithi161.stdout: self.mgr._daemon_action(daemon_spec, action=action) 2024-06-01T02:02:49.363 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 2457, in _daemon_action 2024-06-01T02:02:49.363 INFO:journalctl@ceph.mon.b.smithi161.stdout: CephadmServe(self)._create_daemon(daemon_spec, reconfig=(action == 'reconfig'))) 2024-06-01T02:02:49.363 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 779, in wait_async 2024-06-01T02:02:49.363 INFO:journalctl@ceph.mon.b.smithi161.stdout: return self.event_loop.get_result(coro, timeout) 2024-06-01T02:02:49.363 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/usr/share/ceph/mgr/cephadm/ssh.py", line 136, in get_result 2024-06-01T02:02:49.363 INFO:journalctl@ceph.mon.b.smithi161.stdout: return future.result(timeout) 2024-06-01T02:02:49.363 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 432, in result 2024-06-01T02:02:49.363 INFO:journalctl@ceph.mon.b.smithi161.stdout: return self.__get_result() 2024-06-01T02:02:49.363 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result 2024-06-01T02:02:49.363 INFO:journalctl@ceph.mon.b.smithi161.stdout: raise self._exception 2024-06-01T02:02:49.363 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1377, in _create_daemon 2024-06-01T02:02:49.363 INFO:journalctl@ceph.mon.b.smithi161.stdout: use_current_daemon_image=reconfig, 2024-06-01T02:02:49.364 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1692, in _run_cephadm 2024-06-01T02:02:49.364 INFO:journalctl@ceph.mon.b.smithi161.stdout: f'cephadm exited with an error code: {code}, stderr: {err}') 2024-06-01T02:02:49.364 INFO:journalctl@ceph.mon.b.smithi161.stdout: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi116.gjuxzb ... 2024-06-01T02:02:49.364 INFO:journalctl@ceph.mon.b.smithi161.stdout: Creating ceph-iscsi config... 2024-06-01T02:02:49.364 INFO:journalctl@ceph.mon.b.smithi161.stdout: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/iscsi-gateway.cfg 2024-06-01T02:02:49.364 INFO:journalctl@ceph.mon.b.smithi161.stdout: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/tcmu-runner-entrypoint.sh 2024-06-01T02:02:49.364 INFO:journalctl@ceph.mon.b.smithi161.stdout: Non-zero exit code 1 from systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb 2024-06-01T02:02:49.364 INFO:journalctl@ceph.mon.b.smithi161.stdout: systemctl: stderr Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:02:49.364 INFO:journalctl@ceph.mon.b.smithi161.stdout: systemctl: stderr See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:02:49.364 INFO:journalctl@ceph.mon.b.smithi161.stdout: Traceback (most recent call last): 2024-06-01T02:02:49.364 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-06-01T02:02:49.364 INFO:journalctl@ceph.mon.b.smithi161.stdout: return _run_code(code, main_globals, None, 2024-06-01T02:02:49.364 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-06-01T02:02:49.364 INFO:journalctl@ceph.mon.b.smithi161.stdout: exec(code, run_globals) 2024-06-01T02:02:49.365 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5618, in 2024-06-01T02:02:49.365 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5606, in main 2024-06-01T02:02:49.365 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3096, in command_deploy_from 2024-06-01T02:02:49.365 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3131, in _common_deploy 2024-06-01T02:02:49.365 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:49 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: CherryPy Checker: 2024-06-01T02:02:49.365 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:49 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: The Application mounted at '' has an empty config. 2024-06-01T02:02:49.365 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:49 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: 2024-06-01T02:02:49.365 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:49 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: [01/Jun/2024:02:02:49] ENGINE Serving on http://:::9283 2024-06-01T02:02:49.365 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:49 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[51992]: [01/Jun/2024:02:02:49] ENGINE Bus STARTED 2024-06-01T02:02:49.366 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3151, in _deploy_daemon_container 2024-06-01T02:02:49.366 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 1123, in deploy_daemon 2024-06-01T02:02:49.366 INFO:journalctl@ceph.mon.b.smithi161.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-06-01T02:02:49.366 INFO:journalctl@ceph.mon.b.smithi161.stdout: RuntimeError: Failed command: systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb: Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:02:49.366 INFO:journalctl@ceph.mon.b.smithi161.stdout: See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:02:49.366 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:49 smithi161 ceph-mon[26055]: Reconfiguring prometheus.a (dependencies changed)... 2024-06-01T02:02:49.366 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:49 smithi161 ceph-mon[26055]: Reconfiguring daemon prometheus.a on smithi161 2024-06-01T02:02:49.366 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:49 smithi161 ceph-mon[26055]: Standby manager daemon x started 2024-06-01T02:02:49.366 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:49 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.161:0/4065369440' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-01T02:02:49.366 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:49 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.161:0/4065369440' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T02:02:49.367 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:49 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.161:0/4065369440' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-01T02:02:49.367 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:49 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.161:0/4065369440' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T02:02:49.408 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:49 smithi116 ceph-mon[25082]: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi116.gjuxzb ... 2024-06-01T02:02:49.408 INFO:journalctl@ceph.mon.a.smithi116.stdout: Creating ceph-iscsi config... 2024-06-01T02:02:49.408 INFO:journalctl@ceph.mon.a.smithi116.stdout: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/iscsi-gateway.cfg 2024-06-01T02:02:49.408 INFO:journalctl@ceph.mon.a.smithi116.stdout: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/tcmu-runner-entrypoint.sh 2024-06-01T02:02:49.408 INFO:journalctl@ceph.mon.a.smithi116.stdout: Non-zero exit code 1 from systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb 2024-06-01T02:02:49.408 INFO:journalctl@ceph.mon.a.smithi116.stdout: systemctl: stderr Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:02:49.408 INFO:journalctl@ceph.mon.a.smithi116.stdout: systemctl: stderr See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:02:49.408 INFO:journalctl@ceph.mon.a.smithi116.stdout: Traceback (most recent call last): 2024-06-01T02:02:49.408 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-06-01T02:02:49.408 INFO:journalctl@ceph.mon.a.smithi116.stdout: return _run_code(code, main_globals, None, 2024-06-01T02:02:49.408 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-06-01T02:02:49.408 INFO:journalctl@ceph.mon.a.smithi116.stdout: exec(code, run_globals) 2024-06-01T02:02:49.409 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5618, in 2024-06-01T02:02:49.409 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5606, in main 2024-06-01T02:02:49.409 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3096, in command_deploy_from 2024-06-01T02:02:49.409 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3131, in _common_deploy 2024-06-01T02:02:49.409 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3151, in _deploy_daemon_container 2024-06-01T02:02:49.409 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 1123, in deploy_daemon 2024-06-01T02:02:49.409 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-06-01T02:02:49.409 INFO:journalctl@ceph.mon.a.smithi116.stdout: RuntimeError: Failed command: systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb: Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:02:49.409 INFO:journalctl@ceph.mon.a.smithi116.stdout: See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:02:49.409 INFO:journalctl@ceph.mon.a.smithi116.stdout: Traceback (most recent call last): 2024-06-01T02:02:49.409 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1114, in _check_daemons 2024-06-01T02:02:49.409 INFO:journalctl@ceph.mon.a.smithi116.stdout: self.mgr._daemon_action(daemon_spec, action=action) 2024-06-01T02:02:49.409 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 2457, in _daemon_action 2024-06-01T02:02:49.409 INFO:journalctl@ceph.mon.a.smithi116.stdout: CephadmServe(self)._create_daemon(daemon_spec, reconfig=(action == 'reconfig'))) 2024-06-01T02:02:49.410 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 779, in wait_async 2024-06-01T02:02:49.410 INFO:journalctl@ceph.mon.a.smithi116.stdout: return self.event_loop.get_result(coro, timeout) 2024-06-01T02:02:49.410 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/usr/share/ceph/mgr/cephadm/ssh.py", line 136, in get_result 2024-06-01T02:02:49.410 INFO:journalctl@ceph.mon.a.smithi116.stdout: return future.result(timeout) 2024-06-01T02:02:49.410 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 432, in result 2024-06-01T02:02:49.410 INFO:journalctl@ceph.mon.a.smithi116.stdout: return self.__get_result() 2024-06-01T02:02:49.410 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result 2024-06-01T02:02:49.410 INFO:journalctl@ceph.mon.a.smithi116.stdout: raise self._exception 2024-06-01T02:02:49.410 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1377, in _create_daemon 2024-06-01T02:02:49.410 INFO:journalctl@ceph.mon.a.smithi116.stdout: use_current_daemon_image=reconfig, 2024-06-01T02:02:49.410 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1692, in _run_cephadm 2024-06-01T02:02:49.410 INFO:journalctl@ceph.mon.a.smithi116.stdout: f'cephadm exited with an error code: {code}, stderr: {err}') 2024-06-01T02:02:49.410 INFO:journalctl@ceph.mon.a.smithi116.stdout: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi116.gjuxzb ... 2024-06-01T02:02:49.411 INFO:journalctl@ceph.mon.a.smithi116.stdout: Creating ceph-iscsi config... 2024-06-01T02:02:49.411 INFO:journalctl@ceph.mon.a.smithi116.stdout: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/iscsi-gateway.cfg 2024-06-01T02:02:49.411 INFO:journalctl@ceph.mon.a.smithi116.stdout: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/tcmu-runner-entrypoint.sh 2024-06-01T02:02:49.411 INFO:journalctl@ceph.mon.a.smithi116.stdout: Non-zero exit code 1 from systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb 2024-06-01T02:02:49.411 INFO:journalctl@ceph.mon.a.smithi116.stdout: systemctl: stderr Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:02:49.411 INFO:journalctl@ceph.mon.a.smithi116.stdout: systemctl: stderr See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:02:49.411 INFO:journalctl@ceph.mon.a.smithi116.stdout: Traceback (most recent call last): 2024-06-01T02:02:49.411 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-06-01T02:02:49.411 INFO:journalctl@ceph.mon.a.smithi116.stdout: return _run_code(code, main_globals, None, 2024-06-01T02:02:49.411 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-06-01T02:02:49.411 INFO:journalctl@ceph.mon.a.smithi116.stdout: exec(code, run_globals) 2024-06-01T02:02:49.411 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5618, in 2024-06-01T02:02:49.411 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5606, in main 2024-06-01T02:02:49.411 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3096, in command_deploy_from 2024-06-01T02:02:49.412 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3131, in _common_deploy 2024-06-01T02:02:49.412 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:49 smithi116 ceph-mon[29958]: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi116.gjuxzb ... 2024-06-01T02:02:49.412 INFO:journalctl@ceph.mon.c.smithi116.stdout: Creating ceph-iscsi config... 2024-06-01T02:02:49.412 INFO:journalctl@ceph.mon.c.smithi116.stdout: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/iscsi-gateway.cfg 2024-06-01T02:02:49.412 INFO:journalctl@ceph.mon.c.smithi116.stdout: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/tcmu-runner-entrypoint.sh 2024-06-01T02:02:49.412 INFO:journalctl@ceph.mon.c.smithi116.stdout: Non-zero exit code 1 from systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb 2024-06-01T02:02:49.412 INFO:journalctl@ceph.mon.c.smithi116.stdout: systemctl: stderr Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:02:49.413 INFO:journalctl@ceph.mon.c.smithi116.stdout: systemctl: stderr See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:02:49.413 INFO:journalctl@ceph.mon.c.smithi116.stdout: Traceback (most recent call last): 2024-06-01T02:02:49.413 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-06-01T02:02:49.413 INFO:journalctl@ceph.mon.c.smithi116.stdout: return _run_code(code, main_globals, None, 2024-06-01T02:02:49.413 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-06-01T02:02:49.413 INFO:journalctl@ceph.mon.c.smithi116.stdout: exec(code, run_globals) 2024-06-01T02:02:49.413 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5618, in 2024-06-01T02:02:49.413 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5606, in main 2024-06-01T02:02:49.413 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3096, in command_deploy_from 2024-06-01T02:02:49.413 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3131, in _common_deploy 2024-06-01T02:02:49.413 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3151, in _deploy_daemon_container 2024-06-01T02:02:49.413 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 1123, in deploy_daemon 2024-06-01T02:02:49.413 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-06-01T02:02:49.413 INFO:journalctl@ceph.mon.c.smithi116.stdout: RuntimeError: Failed command: systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb: Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:02:49.414 INFO:journalctl@ceph.mon.c.smithi116.stdout: See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:02:49.414 INFO:journalctl@ceph.mon.c.smithi116.stdout: Traceback (most recent call last): 2024-06-01T02:02:49.414 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1114, in _check_daemons 2024-06-01T02:02:49.414 INFO:journalctl@ceph.mon.c.smithi116.stdout: self.mgr._daemon_action(daemon_spec, action=action) 2024-06-01T02:02:49.414 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 2457, in _daemon_action 2024-06-01T02:02:49.414 INFO:journalctl@ceph.mon.c.smithi116.stdout: CephadmServe(self)._create_daemon(daemon_spec, reconfig=(action == 'reconfig'))) 2024-06-01T02:02:49.414 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 779, in wait_async 2024-06-01T02:02:49.414 INFO:journalctl@ceph.mon.c.smithi116.stdout: return self.event_loop.get_result(coro, timeout) 2024-06-01T02:02:49.414 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/usr/share/ceph/mgr/cephadm/ssh.py", line 136, in get_result 2024-06-01T02:02:49.414 INFO:journalctl@ceph.mon.c.smithi116.stdout: return future.result(timeout) 2024-06-01T02:02:49.414 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 432, in result 2024-06-01T02:02:49.414 INFO:journalctl@ceph.mon.c.smithi116.stdout: return self.__get_result() 2024-06-01T02:02:49.414 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result 2024-06-01T02:02:49.415 INFO:journalctl@ceph.mon.c.smithi116.stdout: raise self._exception 2024-06-01T02:02:49.415 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1377, in _create_daemon 2024-06-01T02:02:49.415 INFO:journalctl@ceph.mon.c.smithi116.stdout: use_current_daemon_image=reconfig, 2024-06-01T02:02:49.415 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1692, in _run_cephadm 2024-06-01T02:02:49.415 INFO:journalctl@ceph.mon.c.smithi116.stdout: f'cephadm exited with an error code: {code}, stderr: {err}') 2024-06-01T02:02:49.415 INFO:journalctl@ceph.mon.c.smithi116.stdout: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi116.gjuxzb ... 2024-06-01T02:02:49.415 INFO:journalctl@ceph.mon.c.smithi116.stdout: Creating ceph-iscsi config... 2024-06-01T02:02:49.415 INFO:journalctl@ceph.mon.c.smithi116.stdout: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/iscsi-gateway.cfg 2024-06-01T02:02:49.415 INFO:journalctl@ceph.mon.c.smithi116.stdout: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/tcmu-runner-entrypoint.sh 2024-06-01T02:02:49.415 INFO:journalctl@ceph.mon.c.smithi116.stdout: Non-zero exit code 1 from systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb 2024-06-01T02:02:49.415 INFO:journalctl@ceph.mon.c.smithi116.stdout: systemctl: stderr Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:02:49.415 INFO:journalctl@ceph.mon.c.smithi116.stdout: systemctl: stderr See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:02:49.415 INFO:journalctl@ceph.mon.c.smithi116.stdout: Traceback (most recent call last): 2024-06-01T02:02:49.416 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-06-01T02:02:49.416 INFO:journalctl@ceph.mon.c.smithi116.stdout: return _run_code(code, main_globals, None, 2024-06-01T02:02:49.416 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-06-01T02:02:49.416 INFO:journalctl@ceph.mon.c.smithi116.stdout: exec(code, run_globals) 2024-06-01T02:02:49.416 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5618, in 2024-06-01T02:02:49.416 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5606, in main 2024-06-01T02:02:49.416 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3096, in command_deploy_from 2024-06-01T02:02:49.416 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3131, in _common_deploy 2024-06-01T02:02:49.416 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3151, in _deploy_daemon_container 2024-06-01T02:02:49.417 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 1123, in deploy_daemon 2024-06-01T02:02:49.417 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-06-01T02:02:49.417 INFO:journalctl@ceph.mon.a.smithi116.stdout: RuntimeError: Failed command: systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb: Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:02:49.417 INFO:journalctl@ceph.mon.a.smithi116.stdout: See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:02:49.417 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:49 smithi116 ceph-mon[25082]: Reconfiguring prometheus.a (dependencies changed)... 2024-06-01T02:02:49.417 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:49 smithi116 ceph-mon[25082]: Reconfiguring daemon prometheus.a on smithi161 2024-06-01T02:02:49.417 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:49 smithi116 ceph-mon[25082]: Standby manager daemon x started 2024-06-01T02:02:49.417 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:49 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.161:0/4065369440' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-01T02:02:49.417 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:49 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.161:0/4065369440' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T02:02:49.417 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:49 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.161:0/4065369440' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-01T02:02:49.417 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:49 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.161:0/4065369440' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T02:02:49.418 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3151, in _deploy_daemon_container 2024-06-01T02:02:49.418 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 1123, in deploy_daemon 2024-06-01T02:02:49.418 INFO:journalctl@ceph.mon.c.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-06-01T02:02:49.418 INFO:journalctl@ceph.mon.c.smithi116.stdout: RuntimeError: Failed command: systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb: Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:02:49.418 INFO:journalctl@ceph.mon.c.smithi116.stdout: See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:02:49.418 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:49 smithi116 ceph-mon[29958]: Reconfiguring prometheus.a (dependencies changed)... 2024-06-01T02:02:49.418 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:49 smithi116 ceph-mon[29958]: Reconfiguring daemon prometheus.a on smithi161 2024-06-01T02:02:49.418 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:49 smithi116 ceph-mon[29958]: Standby manager daemon x started 2024-06-01T02:02:49.418 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:49 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.161:0/4065369440' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-01T02:02:49.418 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:49 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.161:0/4065369440' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T02:02:49.418 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:49 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.161:0/4065369440' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-01T02:02:49.418 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:49 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.161:0/4065369440' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T02:02:50.436 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:50 smithi161 ceph-mon[26055]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 20 op/s 2024-06-01T02:02:50.436 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:50 smithi161 ceph-mon[26055]: mgrmap e30: y(active, since 8s), standbys: x 2024-06-01T02:02:50.436 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:50 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-01T02:02:50.437 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:50 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:50.437 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:50 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:50.437 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:50 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:50.437 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:50 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:50.437 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:50 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:50.437 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:50 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:50.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:50 smithi116 ceph-mon[25082]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 20 op/s 2024-06-01T02:02:50.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:50 smithi116 ceph-mon[25082]: mgrmap e30: y(active, since 8s), standbys: x 2024-06-01T02:02:50.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:50 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-01T02:02:50.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:50 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:50.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:50 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:50.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:50 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:50.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:50 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:50.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:50 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:50.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:50 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:50.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:50 smithi116 ceph-mon[29958]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 20 op/s 2024-06-01T02:02:50.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:50 smithi116 ceph-mon[29958]: mgrmap e30: y(active, since 8s), standbys: x 2024-06-01T02:02:50.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:50 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-01T02:02:50.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:50 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:50.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:50 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:50.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:50 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:50.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:50 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:50.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:50 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:50.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:50 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:51.192 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:51 smithi116 ceph-mon[25082]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-06-01T02:02:51.193 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:51 smithi116 ceph-mon[25082]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-06-01T02:02:51.193 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:51 smithi116 ceph-mon[25082]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 19 op/s 2024-06-01T02:02:51.193 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:51 smithi116 ceph-mon[25082]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s rd, 0 B/s wr, 28 op/s 2024-06-01T02:02:51.193 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:51 smithi116 ceph-mon[29958]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-06-01T02:02:51.193 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:51 smithi116 ceph-mon[29958]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-06-01T02:02:51.193 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:51 smithi116 ceph-mon[29958]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 19 op/s 2024-06-01T02:02:51.193 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:51 smithi116 ceph-mon[29958]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s rd, 0 B/s wr, 28 op/s 2024-06-01T02:02:51.340 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:51 smithi161 ceph-mon[26055]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-06-01T02:02:51.341 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:51 smithi161 ceph-mon[26055]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-06-01T02:02:51.341 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:51 smithi161 ceph-mon[26055]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 19 op/s 2024-06-01T02:02:51.341 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:51 smithi161 ceph-mon[26055]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s rd, 0 B/s wr, 28 op/s 2024-06-01T02:02:52.340 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:52 smithi161 systemd[1]: Stopping Ceph prometheus.a for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:02:52.341 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:52 smithi161 bash[59024]: Error: no container with name or ID "ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus.a" found: no such container 2024-06-01T02:02:52.608 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:52 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:52.344Z caller=main.go:775 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-06-01T02:02:52.608 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:52 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:52.344Z caller=main.go:798 level=info msg="Stopping scrape discovery manager..." 2024-06-01T02:02:52.608 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:52 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:52.344Z caller=main.go:812 level=info msg="Stopping notify discovery manager..." 2024-06-01T02:02:52.608 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:52 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:52.344Z caller=main.go:834 level=info msg="Stopping scrape manager..." 2024-06-01T02:02:52.608 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:52 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:52.344Z caller=main.go:794 level=info msg="Scrape discovery manager stopped" 2024-06-01T02:02:52.608 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:52 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:52.344Z caller=main.go:808 level=info msg="Notify discovery manager stopped" 2024-06-01T02:02:52.608 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:52 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:52.344Z caller=manager.go:945 level=info component="rule manager" msg="Stopping rule manager..." 2024-06-01T02:02:52.608 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:52 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:52.344Z caller=manager.go:955 level=info component="rule manager" msg="Rule manager stopped" 2024-06-01T02:02:52.608 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:52 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:52.344Z caller=main.go:828 level=info msg="Scrape manager stopped" 2024-06-01T02:02:52.608 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:52 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:52.345Z caller=notifier.go:600 level=info component=notifier msg="Stopping notification manager..." 2024-06-01T02:02:52.608 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:52 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:52.345Z caller=main.go:1054 level=info msg="Notifier manager stopped" 2024-06-01T02:02:52.609 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:52 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[54973]: ts=2024-06-01T02:02:52.345Z caller=main.go:1066 level=info msg="See you next time!" 2024-06-01T02:02:52.609 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:52 smithi161 podman[59037]: 2024-06-01 02:02:52.354717269 +0000 UTC m=+0.120211377 container died 1267ec8fa4c91e115570028fadb0074ab5429234244664badbf3a05de353bc95 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-01T02:02:52.945 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:52 smithi161 podman[59037]: 2024-06-01 02:02:52.761079013 +0000 UTC m=+0.526573142 container cleanup 1267ec8fa4c91e115570028fadb0074ab5429234244664badbf3a05de353bc95 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-01T02:02:52.945 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:52 smithi161 bash[59037]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a 2024-06-01T02:02:52.945 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:52 smithi161 bash[59063]: Error: no container with name or ID "ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus.a" found: no such container 2024-06-01T02:02:53.234 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:53 smithi161 ceph-mon[26055]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 38 KiB/s rd, 0 B/s wr, 22 op/s 2024-06-01T02:02:53.234 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:53 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:53.235 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:53 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:53.235 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:53 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:53.235 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:52 smithi161 podman[59051]: 2024-06-01 02:02:52.97099898 +0000 UTC m=+0.615052162 container remove 1267ec8fa4c91e115570028fadb0074ab5429234244664badbf3a05de353bc95 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-01T02:02:53.235 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:52 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-06-01T02:02:53.235 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:52 smithi161 systemd[1]: Stopped Ceph prometheus.a for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:02:53.235 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:52 smithi161 systemd[1]: Starting Ceph prometheus.a for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:02:53.235 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:53 smithi161 podman[59125]: 2024-06-01 02:02:53.110495022 +0000 UTC m=+0.018722686 image pull 514e6a882f6e74806a5856468489eeff8d7106095557578da96935e4d0ba4d9d quay.io/prometheus/prometheus:v2.33.4 2024-06-01T02:02:53.235 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:53 smithi161 podman[59125]: 2024-06-01 02:02:53.238789165 +0000 UTC m=+0.147016852 container create 88d5c4c639cbe3babfdb773f8894b81326c7aa6ad766a2cccf802c40e0ea68c8 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-01T02:02:53.511 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:53 smithi161 podman[59125]: 2024-06-01 02:02:53.440889545 +0000 UTC m=+0.349117190 container init 88d5c4c639cbe3babfdb773f8894b81326c7aa6ad766a2cccf802c40e0ea68c8 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-01T02:02:53.512 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:53 smithi161 podman[59125]: 2024-06-01 02:02:53.443822065 +0000 UTC m=+0.352049719 container start 88d5c4c639cbe3babfdb773f8894b81326c7aa6ad766a2cccf802c40e0ea68c8 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-01T02:02:53.512 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:53 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:02:53.475Z caller=main.go:475 level=info msg="No time or size retention was set so using the default time retention" duration=15d 2024-06-01T02:02:53.512 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:53 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:02:53.475Z caller=main.go:512 level=info msg="Starting Prometheus" version="(version=2.33.4, branch=HEAD, revision=83032011a5d3e6102624fe58241a374a7201fee8)" 2024-06-01T02:02:53.512 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:53 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:02:53.475Z caller=main.go:517 level=info build_context="(go=go1.17.7, user=root@d13bf69e7be8, date=20220222-16:51:28)" 2024-06-01T02:02:53.512 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:53 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:02:53.475Z caller=main.go:518 level=info host_details="(Linux 5.14.0-452.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024 x86_64 smithi161 (none))" 2024-06-01T02:02:53.512 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:53 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:02:53.475Z caller=main.go:519 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-06-01T02:02:53.512 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:53 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:02:53.475Z caller=main.go:520 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-06-01T02:02:53.512 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:53 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:02:53.476Z caller=web.go:570 level=info component=web msg="Start listening for connections" address=:9095 2024-06-01T02:02:53.512 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:53 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:02:53.477Z caller=main.go:923 level=info msg="Starting TSDB ..." 2024-06-01T02:02:53.512 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:53 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:02:53.479Z caller=tls_config.go:195 level=info component=web msg="TLS is disabled." http2=false 2024-06-01T02:02:53.512 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:53 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:02:53.480Z caller=head.go:493 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-06-01T02:02:53.512 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:53 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:02:53.482Z caller=head.go:527 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.213122ms 2024-06-01T02:02:53.513 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:53 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:02:53.482Z caller=head.go:533 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-06-01T02:02:53.513 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:53 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:02:53.483Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=3 2024-06-01T02:02:53.513 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:53 smithi161 bash[59125]: 88d5c4c639cbe3babfdb773f8894b81326c7aa6ad766a2cccf802c40e0ea68c8 2024-06-01T02:02:53.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:53 smithi116 ceph-mon[29958]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 38 KiB/s rd, 0 B/s wr, 22 op/s 2024-06-01T02:02:53.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:53 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:53.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:53 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:53.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:53 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:53.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:53 smithi116 ceph-mon[25082]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail; 38 KiB/s rd, 0 B/s wr, 22 op/s 2024-06-01T02:02:53.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:53 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:53.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:53 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:53.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:53 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:53.763 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:53 smithi161 systemd[1]: Started Ceph prometheus.a for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:02:54.121 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:53 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:02:53.921Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=3 2024-06-01T02:02:54.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:54 smithi161 ceph-mon[26055]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:54.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:54 smithi161 ceph-mon[26055]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:54.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:54 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:54.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:54 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:54.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:54 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-01T02:02:54.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:54 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:02:54.445 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:54 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:02:54.148Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=2 maxSegment=3 2024-06-01T02:02:54.445 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:54 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:02:54.149Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=3 maxSegment=3 2024-06-01T02:02:54.445 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:54 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:02:54.149Z caller=head.go:610 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=37.725µs wal_replay_duration=667.491752ms total_replay_duration=668.759233ms 2024-06-01T02:02:54.445 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:54 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:02:54.155Z caller=main.go:944 level=info fs_type=XFS_SUPER_MAGIC 2024-06-01T02:02:54.446 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:54 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:02:54.155Z caller=main.go:947 level=info msg="TSDB started" 2024-06-01T02:02:54.446 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:54 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:02:54.155Z caller=main.go:1128 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-06-01T02:02:54.446 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:54 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:02:54.175Z caller=main.go:1165 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=19.412496ms db_storage=935ns remote_storage=1.315µs web_handler=360ns query_engine=795ns scrape=480.523µs scrape_sd=150.731µs notify=10.559µs notify_sd=13.397µs rules=18.398002ms 2024-06-01T02:02:54.446 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:02:54 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:02:54.175Z caller=main.go:896 level=info msg="Server is ready to receive web requests." 2024-06-01T02:02:54.497 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:54 smithi116 ceph-mon[25082]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:54.497 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:54 smithi116 ceph-mon[25082]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:54.497 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:54 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:54.497 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:54 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:54.497 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:54 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-01T02:02:54.497 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:54 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:02:54.498 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:54 smithi116 ceph-mon[29958]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:54.498 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:54 smithi116 ceph-mon[29958]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:54.498 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:54 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:54.498 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:54 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:54.498 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:54 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-01T02:02:54.498 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:54 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:02:55.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:55 smithi161 ceph-mon[26055]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-01T02:02:55.235 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:55 smithi161 ceph-mon[26055]: Upgrade: Need to upgrade myself (mgr.y) 2024-06-01T02:02:55.235 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:55 smithi161 ceph-mon[26055]: Upgrade: Need to upgrade myself (mgr.y) 2024-06-01T02:02:55.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:55 smithi116 ceph-mon[25082]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-01T02:02:55.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:55 smithi116 ceph-mon[25082]: Upgrade: Need to upgrade myself (mgr.y) 2024-06-01T02:02:55.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:55 smithi116 ceph-mon[25082]: Upgrade: Need to upgrade myself (mgr.y) 2024-06-01T02:02:55.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:55 smithi116 ceph-mon[29958]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-01T02:02:55.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:55 smithi116 ceph-mon[29958]: Upgrade: Need to upgrade myself (mgr.y) 2024-06-01T02:02:55.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:55 smithi116 ceph-mon[29958]: Upgrade: Need to upgrade myself (mgr.y) 2024-06-01T02:02:56.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:56 smithi161 ceph-mon[26055]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:56.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:56 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:56.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:56 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:56.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:56 smithi161 ceph-mon[26055]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:56.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:56 smithi161 ceph-mon[26055]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:56.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:56 smithi161 ceph-mon[26055]: Upgrade: Updating mgr.x 2024-06-01T02:02:56.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:56 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:02:56.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:56 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-01T02:02:56.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:56 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:56.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:56 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:02:56.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:56 smithi161 ceph-mon[26055]: Deploying daemon mgr.x on smithi161 2024-06-01T02:02:56.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:56 smithi116 ceph-mon[25082]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:56.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:56 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:56.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:56 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:56.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:56 smithi116 ceph-mon[25082]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:56.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:56 smithi116 ceph-mon[25082]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:56.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:56 smithi116 ceph-mon[25082]: Upgrade: Updating mgr.x 2024-06-01T02:02:56.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:56 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:02:56.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:56 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-01T02:02:56.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:56 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:56.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:56 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:02:56.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:56 smithi116 ceph-mon[25082]: Deploying daemon mgr.x on smithi161 2024-06-01T02:02:56.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:56 smithi116 ceph-mon[29958]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:56.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:56 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:56.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:56 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:56.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:56 smithi116 ceph-mon[29958]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:56.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:56 smithi116 ceph-mon[29958]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:56.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:56 smithi116 ceph-mon[29958]: Upgrade: Updating mgr.x 2024-06-01T02:02:56.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:56 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:02:56.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:56 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-01T02:02:56.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:56 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:56.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:56 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:02:56.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:56 smithi116 ceph-mon[29958]: Deploying daemon mgr.x on smithi161 2024-06-01T02:02:57.129 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:57 smithi161 systemd[1]: Stopping Ceph mgr.x for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:02:57.807 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:57 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:57.807 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:57 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:02:57.807 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:02:57 smithi161 ceph-mon[26055]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:57.808 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:57 smithi161 podman[59476]: 2024-06-01 02:02:57.457670518 +0000 UTC m=+0.177852747 container died 2743006903a699506c0f492ca6fdfa50e12b615367d268f02173980e72eca60f (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x, org.label-schema.build-date=20240528, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-06-01T02:02:58.040 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:57 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:58.040 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:57 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:02:58.040 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:02:57 smithi116 ceph-mon[25082]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:58.040 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:57 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:02:58.040 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:57 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:02:58.040 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:02:57 smithi116 ceph-mon[29958]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:02:58.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:57 smithi161 podman[59476]: 2024-06-01 02:02:57.993394615 +0000 UTC m=+0.713576830 container cleanup 2743006903a699506c0f492ca6fdfa50e12b615367d268f02173980e72eca60f (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-01T02:02:58.195 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:57 smithi161 bash[59476]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x 2024-06-01T02:02:58.523 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:58 smithi161 podman[59489]: 2024-06-01 02:02:58.203183849 +0000 UTC m=+0.742696073 container remove 2743006903a699506c0f492ca6fdfa50e12b615367d268f02173980e72eca60f (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x, org.label-schema.build-date=20240528, GIT_CLEAN=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0) 2024-06-01T02:02:58.523 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:58 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-06-01T02:02:58.523 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:58 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mgr.x.service: Failed with result 'exit-code'. 2024-06-01T02:02:58.523 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:58 smithi161 systemd[1]: Stopped Ceph mgr.x for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:02:58.523 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:58 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mgr.x.service: Consumed 28.727s CPU time. 2024-06-01T02:02:58.841 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:58 smithi161 systemd[1]: Starting Ceph mgr.x for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:02:59.153 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:58 smithi161 podman[59622]: 2024-06-01 02:02:58.747994375 +0000 UTC m=+0.017535171 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:02:59.153 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:58 smithi161 podman[59622]: 2024-06-01 02:02:58.84662867 +0000 UTC m=+0.116169484 container create 2d7fbb85e5f748fda5cfc7eb53872a5dba1bfcd91df68f9ef166a946e3f4640a (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-06-01T02:02:59.154 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:59 smithi161 podman[59622]: 2024-06-01 02:02:59.057377238 +0000 UTC m=+0.326918020 container init 2d7fbb85e5f748fda5cfc7eb53872a5dba1bfcd91df68f9ef166a946e3f4640a (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-01T02:02:59.154 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:59 smithi161 podman[59622]: 2024-06-01 02:02:59.060217224 +0000 UTC m=+0.329758006 container start 2d7fbb85e5f748fda5cfc7eb53872a5dba1bfcd91df68f9ef166a946e3f4640a (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-01T02:02:59.154 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:59 smithi161 ceph-mgr[59641]: -- 172.21.15.161:0/1837233167 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5638dedfc5a0 con 0x5638dedd0c00 2024-06-01T02:02:59.154 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:59 smithi161 bash[59622]: 2d7fbb85e5f748fda5cfc7eb53872a5dba1bfcd91df68f9ef166a946e3f4640a 2024-06-01T02:02:59.405 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:59 smithi161 systemd[1]: Started Ceph mgr.x for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:02:59.406 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:59 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:02:59.218+0000 7fcfa46ab200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-01T02:02:59.406 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:59 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:02:59.409+0000 7fcfa46ab200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-01T02:02:59.516 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:02:59 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:02:59] "GET /metrics HTTP/1.1" 200 38241 "" "Prometheus/2.33.4" 2024-06-01T02:02:59.694 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:59 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:02:59.510+0000 7fcfa46ab200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-01T02:03:00.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:59 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:02:59.818+0000 7fcfa46ab200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-01T02:03:00.195 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:02:59 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:02:59.902+0000 7fcfa46ab200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-01T02:03:00.289 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:00 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:03:00.289 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:00 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:03:00.289 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:00 smithi116 ceph-mon[25082]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:00.289 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:00 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:00.289 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:00 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:00.290 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:00 smithi116 ceph-mon[25082]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:00.290 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:00 smithi116 ceph-mon[25082]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:00.290 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:00 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:03:00.290 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:00 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:00.290 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:00 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi116.gjuxzb", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-01T02:03:00.290 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:00 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:00.290 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:00 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/3641441992' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-01T02:03:00.290 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:00 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:03:00.290 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:00 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:03:00.290 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:00 smithi116 ceph-mon[29958]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:00.291 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:00 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:00.291 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:00 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:00.291 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:00 smithi116 ceph-mon[29958]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:00.291 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:00 smithi116 ceph-mon[29958]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:00.291 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:00 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:03:00.291 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:00 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:00.291 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:00 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi116.gjuxzb", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-01T02:03:00.291 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:00 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:00.291 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:00 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/3641441992' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-01T02:03:00.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:00 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:03:00.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:00 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:03:00.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:00 smithi161 ceph-mon[26055]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:00.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:00 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:00.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:00 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:00.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:00 smithi161 ceph-mon[26055]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:00.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:00 smithi161 ceph-mon[26055]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:00.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:00 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' 2024-06-01T02:03:00.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:00 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:00.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:00 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi116.gjuxzb", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-01T02:03:00.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:00 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:00.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:00 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/3641441992' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-01T02:03:01.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:01 smithi116 ceph-mon[25082]: Reconfiguring iscsi.foo.smithi116.gjuxzb (dependencies changed)... 2024-06-01T02:03:01.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:01 smithi116 ceph-mon[25082]: Reconfiguring daemon iscsi.foo.smithi116.gjuxzb on smithi116 2024-06-01T02:03:01.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:01 smithi116 ceph-mon[29958]: Reconfiguring iscsi.foo.smithi116.gjuxzb (dependencies changed)... 2024-06-01T02:03:01.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:01 smithi116 ceph-mon[29958]: Reconfiguring daemon iscsi.foo.smithi116.gjuxzb on smithi116 2024-06-01T02:03:01.433 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:01 smithi161 ceph-mon[26055]: Reconfiguring iscsi.foo.smithi116.gjuxzb (dependencies changed)... 2024-06-01T02:03:01.433 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:01 smithi161 ceph-mon[26055]: Reconfiguring daemon iscsi.foo.smithi116.gjuxzb on smithi116 2024-06-01T02:03:01.433 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:01 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:03:01.096+0000 7fcfa46ab200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-01T02:03:01.433 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:01 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:03:01.265+0000 7fcfa46ab200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-01T02:03:01.434 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:01 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:03:01.437+0000 7fcfa46ab200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-01T02:03:01.949 INFO:teuthology.orchestra.run.smithi116.stdout:true 2024-06-01T02:03:02.019 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:01 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:03:01.728+0000 7fcfa46ab200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-01T02:03:02.020 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:01 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:03:01.820+0000 7fcfa46ab200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-01T02:03:02.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:02 smithi161 ceph-mon[26055]: pgmap v22: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:02.445 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:02 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:03:02.023+0000 7fcfa46ab200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-01T02:03:02.445 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:02 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:03:02.218+0000 7fcfa46ab200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-01T02:03:02.502 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-mon[29958]: pgmap v22: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:02.503 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-mon[25082]: pgmap v22: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:02.616 INFO:teuthology.orchestra.run.smithi116.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-01T02:03:02.616 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi116 smithi116 running 7s ago 11m - - 2024-06-01T02:03:02.616 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi161 smithi161 running 13s ago 11m - - 2024-06-01T02:03:02.616 INFO:teuthology.orchestra.run.smithi116.stdout:alertmanager.a smithi116 *:9093,9094 running (62s) 7s ago 6m 14.3M - 0.23.0 ba2b418f427c e0c297162ded 2024-06-01T02:03:02.616 INFO:teuthology.orchestra.run.smithi116.stdout:grafana.a smithi161 *:3000 running (39s) 13s ago 5m 45.4M - 8.3.5 dad864ee21e9 00b0a78ee88e 2024-06-01T02:03:02.616 INFO:teuthology.orchestra.run.smithi116.stdout:iscsi.foo.smithi116.gjuxzb smithi116 unknown 7s ago 4m - - 2024-06-01T02:03:02.616 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.x smithi161 *:8443,9283,8765 starting - - - - 2024-06-01T02:03:02.616 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.y smithi116 *:8443,9283,8765 running (33s) 7s ago 12m 496M - 19.0.0-2946-g3d89accf b045ac545f82 0af4df0af300 2024-06-01T02:03:02.616 INFO:teuthology.orchestra.run.smithi116.stdout:mon.a smithi116 running (12m) 7s ago 12m 57.6M 2048M 17.2.0 e1d6a67b021e c821092a538a 2024-06-01T02:03:02.617 INFO:teuthology.orchestra.run.smithi116.stdout:mon.b smithi161 running (11m) 13s ago 11m 50.6M 2048M 17.2.0 e1d6a67b021e 9011986aa6e3 2024-06-01T02:03:02.617 INFO:teuthology.orchestra.run.smithi116.stdout:mon.c smithi116 running (11m) 7s ago 11m 51.3M 2048M 17.2.0 e1d6a67b021e 4a6fe0b16d63 2024-06-01T02:03:02.617 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.a smithi116 *:9100 running (70s) 7s ago 6m 8232k - 1.3.1 1dbe0e931976 e347748d17bb 2024-06-01T02:03:02.617 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.b smithi161 *:9100 running (44s) 13s ago 6m 13.1M - 1.3.1 1dbe0e931976 6e66182f5a1a 2024-06-01T02:03:02.617 INFO:teuthology.orchestra.run.smithi116.stdout:osd.0 smithi116 running (10m) 7s ago 10m 53.7M 4096M 17.2.0 e1d6a67b021e 0cc929b0ecad 2024-06-01T02:03:02.617 INFO:teuthology.orchestra.run.smithi116.stdout:osd.1 smithi116 running (10m) 7s ago 10m 54.7M 4096M 17.2.0 e1d6a67b021e 6ff7fc5abe58 2024-06-01T02:03:02.617 INFO:teuthology.orchestra.run.smithi116.stdout:osd.2 smithi116 running (9m) 7s ago 9m 52.0M 4096M 17.2.0 e1d6a67b021e 6aecbb10256b 2024-06-01T02:03:02.617 INFO:teuthology.orchestra.run.smithi116.stdout:osd.3 smithi116 running (9m) 7s ago 9m 53.7M 4096M 17.2.0 e1d6a67b021e a802290f5290 2024-06-01T02:03:02.617 INFO:teuthology.orchestra.run.smithi116.stdout:osd.4 smithi161 running (8m) 13s ago 8m 57.0M 4096M 17.2.0 e1d6a67b021e 70b3f89b0bd9 2024-06-01T02:03:02.617 INFO:teuthology.orchestra.run.smithi116.stdout:osd.5 smithi161 running (8m) 13s ago 8m 51.5M 4096M 17.2.0 e1d6a67b021e 68358b78a25d 2024-06-01T02:03:02.617 INFO:teuthology.orchestra.run.smithi116.stdout:osd.6 smithi161 running (8m) 13s ago 8m 50.2M 4096M 17.2.0 e1d6a67b021e 50abeb0551e7 2024-06-01T02:03:02.617 INFO:teuthology.orchestra.run.smithi116.stdout:osd.7 smithi161 running (7m) 13s ago 7m 54.6M 4096M 17.2.0 e1d6a67b021e 6004e6983735 2024-06-01T02:03:02.617 INFO:teuthology.orchestra.run.smithi116.stdout:prometheus.a smithi161 *:9095 starting - - - - 2024-06-01T02:03:02.617 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi116.vzjrxj smithi116 *:8000 running (5m) 7s ago 5m 74.3M - 17.2.0 e1d6a67b021e df4fe3c1dbc3 2024-06-01T02:03:02.618 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi161.ashqxz smithi161 *:8000 running (5m) 13s ago 5m 75.7M - 17.2.0 e1d6a67b021e 3eecc1f3e2d3 2024-06-01T02:03:02.618 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi116.hokbmf smithi116 *:80 running (4m) 7s ago 4m 74.9M - 17.2.0 e1d6a67b021e 10950ed75dbd 2024-06-01T02:03:02.618 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi161.axjlzw smithi161 *:80 running (4m) 13s ago 4m 73.9M - 17.2.0 e1d6a67b021e 0b5d5ce47772 2024-06-01T02:03:03.025 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:03:03.025 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": { 2024-06-01T02:03:03.025 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 3 2024-06-01T02:03:03.025 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:03:03.025 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": { 2024-06-01T02:03:03.025 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-01T02:03:03.025 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:03:03.026 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": { 2024-06-01T02:03:03.026 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2024-06-01T02:03:03.026 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:03:03.026 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": {}, 2024-06-01T02:03:03.026 INFO:teuthology.orchestra.run.smithi116.stdout: "rgw": { 2024-06-01T02:03:03.026 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-06-01T02:03:03.026 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:03:03.026 INFO:teuthology.orchestra.run.smithi116.stdout: "overall": { 2024-06-01T02:03:03.026 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 15, 2024-06-01T02:03:03.026 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-01T02:03:03.026 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T02:03:03.026 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:03:03.027 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:03:02.824+0000 7ff1eb968700 -1 log_channel(cephadm) log [ERR] : cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi116.gjuxzb ... 2024-06-01T02:03:03.027 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: Creating ceph-iscsi config... 2024-06-01T02:03:03.027 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/iscsi-gateway.cfg 2024-06-01T02:03:03.027 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/tcmu-runner-entrypoint.sh 2024-06-01T02:03:03.027 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: Non-zero exit code 1 from systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb 2024-06-01T02:03:03.028 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: systemctl: stderr Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:03:03.028 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: systemctl: stderr See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:03:03.028 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: Traceback (most recent call last): 2024-06-01T02:03:03.028 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-06-01T02:03:03.028 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: return _run_code(code, main_globals, None, 2024-06-01T02:03:03.028 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-06-01T02:03:03.028 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: exec(code, run_globals) 2024-06-01T02:03:03.028 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5618, in 2024-06-01T02:03:03.028 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5606, in main 2024-06-01T02:03:03.028 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3096, in command_deploy_from 2024-06-01T02:03:03.028 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3131, in _common_deploy 2024-06-01T02:03:03.028 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3151, in _deploy_daemon_container 2024-06-01T02:03:03.028 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 1123, in deploy_daemon 2024-06-01T02:03:03.029 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-06-01T02:03:03.029 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: RuntimeError: Failed command: systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb: Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:03:03.029 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:03:03.029 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: Traceback (most recent call last): 2024-06-01T02:03:03.029 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1114, in _check_daemons 2024-06-01T02:03:03.029 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: self.mgr._daemon_action(daemon_spec, action=action) 2024-06-01T02:03:03.029 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/usr/share/ceph/mgr/cephadm/module.py", line 2457, in _daemon_action 2024-06-01T02:03:03.029 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: CephadmServe(self)._create_daemon(daemon_spec, reconfig=(action == 'reconfig'))) 2024-06-01T02:03:03.029 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/usr/share/ceph/mgr/cephadm/module.py", line 779, in wait_async 2024-06-01T02:03:03.029 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: return self.event_loop.get_result(coro, timeout) 2024-06-01T02:03:03.029 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/usr/share/ceph/mgr/cephadm/ssh.py", line 136, in get_result 2024-06-01T02:03:03.029 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: return future.result(timeout) 2024-06-01T02:03:03.030 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/lib64/python3.6/concurrent/futures/_base.py", line 432, in result 2024-06-01T02:03:03.030 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: return self.__get_result() 2024-06-01T02:03:03.030 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result 2024-06-01T02:03:03.030 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: raise self._exception 2024-06-01T02:03:03.030 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1377, in _create_daemon 2024-06-01T02:03:03.030 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: use_current_daemon_image=reconfig, 2024-06-01T02:03:03.030 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1692, in _run_cephadm 2024-06-01T02:03:03.030 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: f'cephadm exited with an error code: {code}, stderr: {err}') 2024-06-01T02:03:03.030 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi116.gjuxzb ... 2024-06-01T02:03:03.030 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: Creating ceph-iscsi config... 2024-06-01T02:03:03.030 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/iscsi-gateway.cfg 2024-06-01T02:03:03.030 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/tcmu-runner-entrypoint.sh 2024-06-01T02:03:03.030 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: Non-zero exit code 1 from systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb 2024-06-01T02:03:03.031 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: systemctl: stderr Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:03:03.031 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: systemctl: stderr See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:03:03.031 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: Traceback (most recent call last): 2024-06-01T02:03:03.031 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-06-01T02:03:03.031 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: return _run_code(code, main_globals, None, 2024-06-01T02:03:03.031 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-06-01T02:03:03.031 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: exec(code, run_globals) 2024-06-01T02:03:03.031 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5618, in 2024-06-01T02:03:03.031 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5606, in main 2024-06-01T02:03:03.031 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3096, in command_deploy_from 2024-06-01T02:03:03.031 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3131, in _common_deploy 2024-06-01T02:03:03.031 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3151, in _deploy_daemon_container 2024-06-01T02:03:03.032 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 1123, in deploy_daemon 2024-06-01T02:03:03.032 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-06-01T02:03:03.032 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: RuntimeError: Failed command: systemctl restart ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb: Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:03:03.032 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:03:03.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:02 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:03:02.827+0000 7fcfa46ab200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-01T02:03:03.277 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:03 smithi116 ceph-mon[29958]: from='client.25336 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:03:03.277 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:03 smithi116 ceph-mon[29958]: from='client.25342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:03:03.277 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:03 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:03.277 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:03 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-06-01T02:03:03.277 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:03 smithi116 ceph-mon[29958]: osdmap e106: 8 total, 8 up, 8 in 2024-06-01T02:03:03.277 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:03 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/3545675466' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:03.278 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:03 smithi116 ceph-mon[25082]: from='client.25336 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:03:03.278 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:03 smithi116 ceph-mon[25082]: from='client.25342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:03:03.278 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:03 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:03.278 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:03 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-06-01T02:03:03.278 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:03 smithi116 ceph-mon[25082]: osdmap e106: 8 total, 8 up, 8 in 2024-06-01T02:03:03.278 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:03 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/3545675466' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:03.278 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:03 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:03:03.249+0000 7ff2593d3700 -1 mgr handle_mgr_map I was active but no longer am 2024-06-01T02:03:03.501 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:03 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:03:03.323+0000 7fcfa46ab200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-01T02:03:03.501 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:03 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:03:03.436+0000 7fcfa46ab200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-01T02:03:03.501 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:03 smithi161 ceph-mon[26055]: from='client.25336 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:03:03.502 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:03 smithi161 ceph-mon[26055]: from='client.25342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:03:03.502 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:03 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:03.502 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:03 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-06-01T02:03:03.502 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:03 smithi161 ceph-mon[26055]: osdmap e106: 8 total, 8 up, 8 in 2024-06-01T02:03:03.502 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:03 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/3545675466' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:03.543 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:03 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: ignoring --setuser ceph since I am not root 2024-06-01T02:03:03.543 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:03 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: ignoring --setgroup ceph since I am not root 2024-06-01T02:03:03.543 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:03 smithi116 ceph-mgr[66344]: -- 172.21.15.116:0/2497771937 <== mon.0 v2:172.21.15.116:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56147f2785a0 con 0x56147f24cc00 2024-06-01T02:03:03.543 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:03 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:03:03.429+0000 7f7ededc1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-01T02:03:03.855 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:03 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:03:03.662+0000 7fcfa46ab200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-01T02:03:03.855 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:03 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:03:03.505Z caller=manager.go:609 level=warn component="rule manager" group=nodes msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on(instance) group_left(nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi116\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:03:04.020 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:03 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:03:03.612+0000 7f7ededc1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-01T02:03:04.021 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:03 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:03:03.715+0000 7f7ededc1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-01T02:03:04.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:04 smithi116 ceph-mon[29958]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-06-01T02:03:04.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:04 smithi116 ceph-mon[29958]: mgrmap e31: x(active, starting, since 0.41353s) 2024-06-01T02:03:04.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:04 smithi116 ceph-mon[25082]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-06-01T02:03:04.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:04 smithi116 ceph-mon[25082]: mgrmap e31: x(active, starting, since 0.41353s) 2024-06-01T02:03:04.294 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:04 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:03:04.024+0000 7f7ededc1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-01T02:03:04.294 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:04 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:03:04.111+0000 7f7ededc1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-01T02:03:04.525 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:04 smithi161 ceph-mon[26055]: from='mgr.15330 172.21.15.116:0/1008436393' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-06-01T02:03:04.525 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:04 smithi161 ceph-mon[26055]: mgrmap e31: x(active, starting, since 0.41353s) 2024-06-01T02:03:04.525 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:03:04.229+0000 7fcfa46ab200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-01T02:03:04.525 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:03:04.338+0000 7fcfa46ab200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-01T02:03:04.842 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:03:04.597+0000 7fcfa46ab200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-01T02:03:04.843 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:03:04.745+0000 7fcfa46ab200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-01T02:03:05.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:03:04.846+0000 7fcfa46ab200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-01T02:03:05.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:03:04.966+0000 7fcfa46ab200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-01T02:03:05.495 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:05 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:03:05.348+0000 7f7ededc1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-01T02:03:05.607 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:03:05.305+0000 7fcfa46ab200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-01T02:03:05.793 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:05 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:03:05.534+0000 7f7ededc1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-01T02:03:05.793 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:05 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:03:05.707+0000 7f7ededc1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-01T02:03:06.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:03:05.944+0000 7fcfa46ab200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-01T02:03:06.195 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:03:06.187+0000 7fcfa46ab200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-01T02:03:06.195 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: [01/Jun/2024:02:03:06] ENGINE Bus STARTING 2024-06-01T02:03:06.293 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:05 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:03:05.998+0000 7f7ededc1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-01T02:03:06.293 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:06 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:03:06.091+0000 7f7ededc1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-01T02:03:06.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:06 smithi116 ceph-mon[25082]: Active manager daemon x restarted 2024-06-01T02:03:06.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:06 smithi116 ceph-mon[25082]: Activating manager daemon x 2024-06-01T02:03:06.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:06 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-01T02:03:06.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:06 smithi116 ceph-mon[25082]: osdmap e107: 8 total, 8 up, 8 in 2024-06-01T02:03:06.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:06 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T02:03:06.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:06 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-01T02:03:06.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:06 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T02:03:06.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:06 smithi116 ceph-mon[29958]: Active manager daemon x restarted 2024-06-01T02:03:06.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:06 smithi116 ceph-mon[29958]: Activating manager daemon x 2024-06-01T02:03:06.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:06 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-01T02:03:06.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:06 smithi116 ceph-mon[29958]: osdmap e107: 8 total, 8 up, 8 in 2024-06-01T02:03:06.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:06 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T02:03:06.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:06 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-01T02:03:06.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:06 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T02:03:06.525 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:06 smithi161 ceph-mon[26055]: Active manager daemon x restarted 2024-06-01T02:03:06.526 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:06 smithi161 ceph-mon[26055]: Activating manager daemon x 2024-06-01T02:03:06.526 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:06 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-01T02:03:06.526 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:06 smithi161 ceph-mon[26055]: osdmap e107: 8 total, 8 up, 8 in 2024-06-01T02:03:06.526 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:06 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T02:03:06.526 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:06 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-01T02:03:06.526 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:06 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T02:03:06.526 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: CherryPy Checker: 2024-06-01T02:03:06.526 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: The Application mounted at '' has an empty config. 2024-06-01T02:03:06.526 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:03:06.527 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: [01/Jun/2024:02:03:06] ENGINE Serving on http://:::9283 2024-06-01T02:03:06.527 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: [01/Jun/2024:02:03:06] ENGINE Bus STARTED 2024-06-01T02:03:06.527 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: [01/Jun/2024:02:03:06] ENGINE Bus STOPPING 2024-06-01T02:03:06.527 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: [01/Jun/2024:02:03:06] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-06-01T02:03:06.527 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: [01/Jun/2024:02:03:06] ENGINE Bus STOPPED 2024-06-01T02:03:06.527 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: [01/Jun/2024:02:03:06] ENGINE Bus STARTING 2024-06-01T02:03:06.527 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: [01/Jun/2024:02:03:06] ENGINE Serving on http://:::9283 2024-06-01T02:03:06.793 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:06 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:03:06.300+0000 7f7ededc1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-01T02:03:06.793 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:06 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:03:06.505+0000 7f7ededc1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-01T02:03:06.817 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: [01/Jun/2024:02:03:06] ENGINE Bus STARTED 2024-06-01T02:03:07.191 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:03:07.146+0000 7f7ededc1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-01T02:03:07.277 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:03:07.277 INFO:teuthology.orchestra.run.smithi116.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-01T02:03:07.277 INFO:teuthology.orchestra.run.smithi116.stdout: "in_progress": true, 2024-06-01T02:03:07.277 INFO:teuthology.orchestra.run.smithi116.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-01T02:03:07.277 INFO:teuthology.orchestra.run.smithi116.stdout: "services_complete": [], 2024-06-01T02:03:07.277 INFO:teuthology.orchestra.run.smithi116.stdout: "progress": "", 2024-06-01T02:03:07.277 INFO:teuthology.orchestra.run.smithi116.stdout: "message": "", 2024-06-01T02:03:07.278 INFO:teuthology.orchestra.run.smithi116.stdout: "is_paused": false 2024-06-01T02:03:07.278 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:03:07.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: mgrmap e32: x(active, starting, since 0.0557933s) 2024-06-01T02:03:07.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T02:03:07.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T02:03:07.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T02:03:07.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-01T02:03:07.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T02:03:07.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T02:03:07.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T02:03:07.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T02:03:07.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T02:03:07.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T02:03:07.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T02:03:07.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T02:03:07.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-01T02:03:07.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-01T02:03:07.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-01T02:03:07.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: Manager daemon x is now available 2024-06-01T02:03:07.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:07.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:03:07.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:03:07.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:03:07.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:03:07.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:03:07.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-01T02:03:07.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-01T02:03:07.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-01T02:03:07.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-01T02:03:07.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:03:07.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:03:07.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi116", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:03:07.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi116", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:03:07.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:07.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:07.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: Updating smithi116:/etc/ceph/ceph.conf 2024-06-01T02:03:07.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: Updating smithi161:/etc/ceph/ceph.conf 2024-06-01T02:03:07.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:07 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:07.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: mgrmap e32: x(active, starting, since 0.0557933s) 2024-06-01T02:03:07.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T02:03:07.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T02:03:07.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T02:03:07.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-01T02:03:07.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T02:03:07.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T02:03:07.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T02:03:07.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T02:03:07.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T02:03:07.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T02:03:07.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T02:03:07.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T02:03:07.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-01T02:03:07.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-01T02:03:07.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-01T02:03:07.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: Manager daemon x is now available 2024-06-01T02:03:07.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:07.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:03:07.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:03:07.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:03:07.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:03:07.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:03:07.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-01T02:03:07.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-01T02:03:07.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-01T02:03:07.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: mgrmap e32: x(active, starting, since 0.0557933s) 2024-06-01T02:03:07.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T02:03:07.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T02:03:07.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T02:03:07.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-01T02:03:07.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T02:03:07.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T02:03:07.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T02:03:07.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T02:03:07.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T02:03:07.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T02:03:07.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T02:03:07.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T02:03:07.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-01T02:03:07.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-01T02:03:07.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-01T02:03:07.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: Manager daemon x is now available 2024-06-01T02:03:07.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:07.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:03:07.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:03:07.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:03:07.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:03:07.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:03:07.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-01T02:03:07.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-01T02:03:07.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-01T02:03:07.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-01T02:03:07.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:03:07.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:03:07.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi116", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:03:07.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi116", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:03:07.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:07.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:07.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: Updating smithi116:/etc/ceph/ceph.conf 2024-06-01T02:03:07.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: Updating smithi161:/etc/ceph/ceph.conf 2024-06-01T02:03:07.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:07.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-01T02:03:07.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:03:07.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:03:07.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi116", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:03:07.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi116", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:03:07.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:07.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:07.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: Updating smithi116:/etc/ceph/ceph.conf 2024-06-01T02:03:07.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: Updating smithi161:/etc/ceph/ceph.conf 2024-06-01T02:03:07.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:07.699 INFO:teuthology.orchestra.run.smithi116.stdout:HEALTH_WARN 1 failed cephadm daemon(s) 2024-06-01T02:03:07.700 INFO:teuthology.orchestra.run.smithi116.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-06-01T02:03:07.700 INFO:teuthology.orchestra.run.smithi116.stdout: daemon iscsi.foo.smithi116.gjuxzb on smithi116 is in unknown state 2024-06-01T02:03:07.957 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:03:07.649+0000 7f7ededc1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-01T02:03:07.957 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:03:07.755+0000 7f7ededc1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-01T02:03:08.262 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:07 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:03:07.974+0000 7f7ededc1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-01T02:03:08.494 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:08 smithi161 ceph-mon[26055]: mgrmap e33: x(active, since 1.0768s) 2024-06-01T02:03:08.495 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:08 smithi161 ceph-mon[26055]: from='client.25360 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:03:08.495 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:08 smithi161 ceph-mon[26055]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:08.495 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:08 smithi161 ceph-mon[26055]: Updating smithi116:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-01T02:03:08.495 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:08 smithi161 ceph-mon[26055]: Updating smithi161:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-01T02:03:08.495 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:08 smithi161 ceph-mon[26055]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-06-01T02:03:08.495 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:08 smithi161 ceph-mon[26055]: Updating smithi116:/etc/ceph/ceph.client.admin.keyring 2024-06-01T02:03:08.495 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:08 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/2239035536' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:03:08.495 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:08 smithi161 ceph-mon[26055]: [01/Jun/2024:02:03:07] ENGINE Bus STARTING 2024-06-01T02:03:08.495 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:08 smithi161 ceph-mon[26055]: [01/Jun/2024:02:03:07] ENGINE Serving on https://172.21.15.161:7151 2024-06-01T02:03:08.495 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:08 smithi161 ceph-mon[26055]: Updating smithi116:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-01T02:03:08.495 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:08 smithi161 ceph-mon[26055]: Updating smithi161:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-01T02:03:08.495 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:08 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:08.496 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:08 smithi161 ceph-mon[26055]: [01/Jun/2024:02:03:07] ENGINE Serving on http://172.21.15.161:8765 2024-06-01T02:03:08.496 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:08 smithi161 ceph-mon[26055]: [01/Jun/2024:02:03:07] ENGINE Bus STARTED 2024-06-01T02:03:08.496 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:08 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:08.496 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:08 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:08.496 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:08 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:08.496 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:08 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:08.496 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:08 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:08.496 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:08 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:08.532 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[29958]: mgrmap e33: x(active, since 1.0768s) 2024-06-01T02:03:08.532 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[29958]: from='client.25360 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:03:08.532 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[29958]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:08.532 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[29958]: Updating smithi116:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-01T02:03:08.532 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[29958]: Updating smithi161:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-01T02:03:08.533 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[29958]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-06-01T02:03:08.533 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[29958]: Updating smithi116:/etc/ceph/ceph.client.admin.keyring 2024-06-01T02:03:08.533 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/2239035536' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:03:08.533 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[29958]: [01/Jun/2024:02:03:07] ENGINE Bus STARTING 2024-06-01T02:03:08.533 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[29958]: [01/Jun/2024:02:03:07] ENGINE Serving on https://172.21.15.161:7151 2024-06-01T02:03:08.533 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[29958]: Updating smithi116:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-01T02:03:08.533 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[29958]: Updating smithi161:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-01T02:03:08.533 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:08.533 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[29958]: [01/Jun/2024:02:03:07] ENGINE Serving on http://172.21.15.161:8765 2024-06-01T02:03:08.533 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[29958]: [01/Jun/2024:02:03:07] ENGINE Bus STARTED 2024-06-01T02:03:08.533 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:08.533 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:08.533 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:08.533 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:08.534 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:08.534 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:08.534 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[25082]: mgrmap e33: x(active, since 1.0768s) 2024-06-01T02:03:08.534 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[25082]: from='client.25360 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:03:08.534 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[25082]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:08.534 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[25082]: Updating smithi116:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-01T02:03:08.534 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[25082]: Updating smithi161:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-01T02:03:08.534 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[25082]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-06-01T02:03:08.534 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[25082]: Updating smithi116:/etc/ceph/ceph.client.admin.keyring 2024-06-01T02:03:08.534 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/2239035536' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:03:08.535 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[25082]: [01/Jun/2024:02:03:07] ENGINE Bus STARTING 2024-06-01T02:03:08.535 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[25082]: [01/Jun/2024:02:03:07] ENGINE Serving on https://172.21.15.161:7151 2024-06-01T02:03:08.535 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[25082]: Updating smithi116:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-01T02:03:08.535 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[25082]: Updating smithi161:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-01T02:03:08.535 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:08.535 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[25082]: [01/Jun/2024:02:03:07] ENGINE Serving on http://172.21.15.161:8765 2024-06-01T02:03:08.535 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[25082]: [01/Jun/2024:02:03:07] ENGINE Bus STARTED 2024-06-01T02:03:08.535 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:08.535 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:08.535 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:08.535 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:08.535 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:08.535 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:08.778 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:08 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:03:08.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"172.21.15.161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"smithi116:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:03:08.793 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:03:08.535+0000 7f7ededc1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-01T02:03:08.793 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:03:08.631+0000 7f7ededc1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-01T02:03:09.263 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:08 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:03:08.924+0000 7f7ededc1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-01T02:03:09.263 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:09 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:03:09.071+0000 7f7ededc1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-01T02:03:09.263 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:09 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:03:09.174+0000 7f7ededc1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-01T02:03:09.334 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:09 smithi161 ceph-mon[26055]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:09.334 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:09 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:09.334 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:03:09.334 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:09 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:03:09.334 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:09 smithi161 ceph-mon[26055]: mgrmap e34: x(active, since 2s) 2024-06-01T02:03:09.543 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:09 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:03:09.301+0000 7f7ededc1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-01T02:03:09.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:09 smithi116 ceph-mon[29958]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:09.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:09 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:09.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:03:09.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:09 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:03:09.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:09 smithi116 ceph-mon[29958]: mgrmap e34: x(active, since 2s) 2024-06-01T02:03:09.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:09 smithi116 ceph-mon[25082]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:09.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:09 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:09.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:03:09.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:09 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:03:09.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:09 smithi116 ceph-mon[25082]: mgrmap e34: x(active, since 2s) 2024-06-01T02:03:09.987 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:09 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:03:09.620+0000 7f7ededc1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-01T02:03:10.522 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:03:10.522 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:10 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:03:10.522 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:10 smithi116 ceph-mon[25082]: Reconfiguring prometheus.a (dependencies changed)... 2024-06-01T02:03:10.522 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:10 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:10.522 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:10 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:10.522 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:10 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:10.523 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:10 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:10.523 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:10 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:10.523 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:10 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:10.523 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:10 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:10.523 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:03:10.523 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:10 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:03:10.523 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:10 smithi116 ceph-mon[29958]: Reconfiguring prometheus.a (dependencies changed)... 2024-06-01T02:03:10.523 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:10 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:10.523 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:10 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:10.523 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:10 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:10.523 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:10 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:10.524 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:10 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:10.524 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:10 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:10.524 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:10 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:10.524 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:10 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:03:10.287+0000 7f7ededc1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-01T02:03:10.537 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:03:10.537 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:10 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:03:10.537 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:10 smithi161 ceph-mon[26055]: Reconfiguring prometheus.a (dependencies changed)... 2024-06-01T02:03:10.537 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:10 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:10.537 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:10 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:10.538 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:10 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:10.538 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:10 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:10.538 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:10 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:10.538 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:10 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:10.538 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:10 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:10.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:10 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:03:10.525+0000 7f7ededc1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-01T02:03:10.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:10 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: [01/Jun/2024:02:03:10] ENGINE Bus STARTING 2024-06-01T02:03:10.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:10 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: CherryPy Checker: 2024-06-01T02:03:10.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:10 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: The Application mounted at '' has an empty config. 2024-06-01T02:03:10.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:10 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: 2024-06-01T02:03:10.795 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:10 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: [01/Jun/2024:02:03:10] ENGINE Serving on http://:::9283 2024-06-01T02:03:10.795 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:10 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: [01/Jun/2024:02:03:10] ENGINE Bus STARTED 2024-06-01T02:03:11.436 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[25082]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:11.436 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[25082]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:11.436 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[25082]: Reconfiguring daemon prometheus.a on smithi161 2024-06-01T02:03:11.436 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[25082]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:11.436 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[25082]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:11.436 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[25082]: Standby manager daemon y started 2024-06-01T02:03:11.436 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.116:0/219883994' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-01T02:03:11.436 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.116:0/219883994' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T02:03:11.436 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.116:0/219883994' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-01T02:03:11.436 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.116:0/219883994' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T02:03:11.436 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:11.436 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:11.437 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:11.437 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[25082]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:11.437 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[25082]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:11.437 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:11.437 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[25082]: mgrmap e35: x(active, since 4s), standbys: y 2024-06-01T02:03:11.437 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-01T02:03:11.437 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[29958]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:11.437 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[29958]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:11.437 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[29958]: Reconfiguring daemon prometheus.a on smithi161 2024-06-01T02:03:11.437 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[29958]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:11.437 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[29958]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:11.438 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[29958]: Standby manager daemon y started 2024-06-01T02:03:11.438 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.116:0/219883994' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-01T02:03:11.438 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.116:0/219883994' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T02:03:11.438 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.116:0/219883994' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-01T02:03:11.438 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.116:0/219883994' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T02:03:11.438 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:11.438 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:11.438 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:11.438 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[29958]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:11.438 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[29958]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:11.438 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:11.438 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[29958]: mgrmap e35: x(active, since 4s), standbys: y 2024-06-01T02:03:11.438 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-01T02:03:11.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:11 smithi161 ceph-mon[26055]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:11.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:11 smithi161 ceph-mon[26055]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:11.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:11 smithi161 ceph-mon[26055]: Reconfiguring daemon prometheus.a on smithi161 2024-06-01T02:03:11.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:11 smithi161 ceph-mon[26055]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:11.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:11 smithi161 ceph-mon[26055]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:11.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:11 smithi161 ceph-mon[26055]: Standby manager daemon y started 2024-06-01T02:03:11.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:11 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.116:0/219883994' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-01T02:03:11.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:11 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.116:0/219883994' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T02:03:11.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:11 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.116:0/219883994' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-01T02:03:11.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:11 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.116:0/219883994' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T02:03:11.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:11 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:11.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:11 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:11.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:11 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:11.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:11 smithi161 ceph-mon[26055]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:11.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:11 smithi161 ceph-mon[26055]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:11.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:11 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:11.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:11 smithi161 ceph-mon[26055]: mgrmap e35: x(active, since 4s), standbys: y 2024-06-01T02:03:11.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-01T02:03:12.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:12 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:12.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:12 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:12.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:12 smithi161 ceph-mon[26055]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:12.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:12 smithi161 ceph-mon[26055]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:12.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:12 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:12.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:12 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:12.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:12 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:12.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:12 smithi161 ceph-mon[26055]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:12.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:12 smithi161 ceph-mon[26055]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:12.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:12 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:12.996 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:12 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:12.996 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:12 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:12.996 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:12 smithi116 ceph-mon[25082]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:12.996 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:12 smithi116 ceph-mon[25082]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:12.997 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:12 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:12.997 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:12 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:12.997 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:12 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:12.997 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:12 smithi116 ceph-mon[25082]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:12.997 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:12 smithi116 ceph-mon[25082]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:12.997 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:12 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:12.997 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:12 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:12.997 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:12 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:12.997 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:12 smithi116 ceph-mon[29958]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:12.997 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:12 smithi116 ceph-mon[29958]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:12.997 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:12 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:12.998 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:12 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:12.998 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:12 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:12.998 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:12 smithi116 ceph-mon[29958]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:12.998 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:12 smithi116 ceph-mon[29958]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 98 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:12.998 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:12 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:13.945 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:13 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:03:13.504Z caller=manager.go:609 level=warn component="rule manager" group=nodes msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on(instance) group_left(nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi116\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:03:14.287 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:14 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:14.287 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:14 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:14.287 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:14 smithi116 ceph-mon[25082]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 93 KiB/s rd, 0 B/s wr, 52 op/s 2024-06-01T02:03:14.287 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:14 smithi116 ceph-mon[25082]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 93 KiB/s rd, 0 B/s wr, 52 op/s 2024-06-01T02:03:14.288 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:14 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:14.288 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:14 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:14.288 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:14 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:14.288 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:14 smithi116 ceph-mon[29958]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 93 KiB/s rd, 0 B/s wr, 52 op/s 2024-06-01T02:03:14.288 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:14 smithi116 ceph-mon[29958]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 93 KiB/s rd, 0 B/s wr, 52 op/s 2024-06-01T02:03:14.288 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:14 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:14.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:14 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:14.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:14 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:14.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:14 smithi161 ceph-mon[26055]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 93 KiB/s rd, 0 B/s wr, 52 op/s 2024-06-01T02:03:14.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:14 smithi161 ceph-mon[26055]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 93 KiB/s rd, 0 B/s wr, 52 op/s 2024-06-01T02:03:14.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:14 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:16.640 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:16 smithi116 ceph-mon[25082]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s rd, 0 B/s wr, 33 op/s 2024-06-01T02:03:16.640 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:16 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/411831784' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-01T02:03:16.641 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:16 smithi116 ceph-mon[29958]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s rd, 0 B/s wr, 33 op/s 2024-06-01T02:03:16.641 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:16 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/411831784' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-01T02:03:16.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:16 smithi161 ceph-mon[26055]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s rd, 0 B/s wr, 33 op/s 2024-06-01T02:03:16.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:16 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/411831784' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-01T02:03:17.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:17 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:17.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:17 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:17.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:17 smithi161 ceph-mon[26055]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 0 B/s wr, 23 op/s 2024-06-01T02:03:17.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:17 smithi161 ceph-mon[26055]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 45 KiB/s rd, 0 B/s wr, 25 op/s 2024-06-01T02:03:17.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:17 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:17.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:17 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/173324417' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1220024770"}]: dispatch 2024-06-01T02:03:18.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:17 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:18.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:17 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:18.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:17 smithi116 ceph-mon[29958]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 0 B/s wr, 23 op/s 2024-06-01T02:03:18.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:17 smithi116 ceph-mon[29958]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 45 KiB/s rd, 0 B/s wr, 25 op/s 2024-06-01T02:03:18.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:17 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:18.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:17 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/173324417' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1220024770"}]: dispatch 2024-06-01T02:03:18.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:17 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:18.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:17 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:18.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:17 smithi116 ceph-mon[25082]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 0 B/s wr, 23 op/s 2024-06-01T02:03:18.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:17 smithi116 ceph-mon[25082]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 45 KiB/s rd, 0 B/s wr, 25 op/s 2024-06-01T02:03:18.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:17 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:18.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:17 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/173324417' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1220024770"}]: dispatch 2024-06-01T02:03:18.915 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:18 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/173324417' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1220024770"}]': finished 2024-06-01T02:03:18.916 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:18 smithi161 ceph-mon[26055]: osdmap e108: 8 total, 8 up, 8 in 2024-06-01T02:03:18.916 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:18 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/4169682155' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3924758188"}]: dispatch 2024-06-01T02:03:19.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:18 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/173324417' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1220024770"}]': finished 2024-06-01T02:03:19.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:18 smithi116 ceph-mon[25082]: osdmap e108: 8 total, 8 up, 8 in 2024-06-01T02:03:19.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:18 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/4169682155' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3924758188"}]: dispatch 2024-06-01T02:03:19.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:18 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/173324417' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1220024770"}]': finished 2024-06-01T02:03:19.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:18 smithi116 ceph-mon[29958]: osdmap e108: 8 total, 8 up, 8 in 2024-06-01T02:03:19.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:18 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/4169682155' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3924758188"}]: dispatch 2024-06-01T02:03:19.647 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:19 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[66280]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:03:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-06-01T02:03:19.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:19 smithi161 ceph-mon[26055]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 372 B/s rd, 0 op/s 2024-06-01T02:03:19.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:19 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/4169682155' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3924758188"}]': finished 2024-06-01T02:03:19.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:19 smithi161 ceph-mon[26055]: osdmap e109: 8 total, 8 up, 8 in 2024-06-01T02:03:19.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:19 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/821819084' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/2654845353"}]: dispatch 2024-06-01T02:03:20.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:19 smithi116 ceph-mon[25082]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 372 B/s rd, 0 op/s 2024-06-01T02:03:20.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:19 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/4169682155' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3924758188"}]': finished 2024-06-01T02:03:20.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:19 smithi116 ceph-mon[25082]: osdmap e109: 8 total, 8 up, 8 in 2024-06-01T02:03:20.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:19 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/821819084' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/2654845353"}]: dispatch 2024-06-01T02:03:20.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:19 smithi116 ceph-mon[29958]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 372 B/s rd, 0 op/s 2024-06-01T02:03:20.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:19 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/4169682155' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/3924758188"}]': finished 2024-06-01T02:03:20.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:19 smithi116 ceph-mon[29958]: osdmap e109: 8 total, 8 up, 8 in 2024-06-01T02:03:20.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:19 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/821819084' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/2654845353"}]: dispatch 2024-06-01T02:03:20.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:20 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/821819084' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/2654845353"}]': finished 2024-06-01T02:03:20.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:20 smithi161 ceph-mon[26055]: osdmap e110: 8 total, 8 up, 8 in 2024-06-01T02:03:20.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:20 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/3760274103' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1076742300"}]: dispatch 2024-06-01T02:03:21.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:20 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/821819084' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/2654845353"}]': finished 2024-06-01T02:03:21.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:20 smithi116 ceph-mon[25082]: osdmap e110: 8 total, 8 up, 8 in 2024-06-01T02:03:21.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:20 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/3760274103' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1076742300"}]: dispatch 2024-06-01T02:03:21.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:20 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/821819084' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6800/2654845353"}]': finished 2024-06-01T02:03:21.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:20 smithi116 ceph-mon[29958]: osdmap e110: 8 total, 8 up, 8 in 2024-06-01T02:03:21.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:20 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/3760274103' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1076742300"}]: dispatch 2024-06-01T02:03:21.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:21 smithi161 ceph-mon[26055]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-06-01T02:03:21.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:21 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/3760274103' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1076742300"}]': finished 2024-06-01T02:03:21.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:21 smithi161 ceph-mon[26055]: osdmap e111: 8 total, 8 up, 8 in 2024-06-01T02:03:21.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:21 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/3413543143' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/2654845353"}]: dispatch 2024-06-01T02:03:21.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:21 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:03:21.946 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:21 smithi161 systemd[1]: Stopping Ceph prometheus.a for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:03:21.946 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:21 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:03:21.783Z caller=main.go:775 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-06-01T02:03:21.946 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:21 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:03:21.783Z caller=main.go:798 level=info msg="Stopping scrape discovery manager..." 2024-06-01T02:03:21.946 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:21 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:03:21.783Z caller=main.go:812 level=info msg="Stopping notify discovery manager..." 2024-06-01T02:03:21.946 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:21 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:03:21.783Z caller=main.go:834 level=info msg="Stopping scrape manager..." 2024-06-01T02:03:21.946 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:21 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:03:21.783Z caller=main.go:794 level=info msg="Scrape discovery manager stopped" 2024-06-01T02:03:21.946 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:21 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:03:21.783Z caller=main.go:808 level=info msg="Notify discovery manager stopped" 2024-06-01T02:03:21.946 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:21 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:03:21.783Z caller=manager.go:945 level=info component="rule manager" msg="Stopping rule manager..." 2024-06-01T02:03:21.946 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:21 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:03:21.783Z caller=main.go:828 level=info msg="Scrape manager stopped" 2024-06-01T02:03:21.946 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:21 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:03:21.783Z caller=manager.go:955 level=info component="rule manager" msg="Rule manager stopped" 2024-06-01T02:03:21.946 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:21 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:03:21.783Z caller=notifier.go:600 level=info component=notifier msg="Stopping notification manager..." 2024-06-01T02:03:21.946 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:21 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:03:21.783Z caller=main.go:1054 level=info msg="Notifier manager stopped" 2024-06-01T02:03:21.947 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:21 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[59140]: ts=2024-06-01T02:03:21.784Z caller=main.go:1066 level=info msg="See you next time!" 2024-06-01T02:03:21.947 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:21 smithi161 podman[62245]: 2024-06-01 02:03:21.793511459 +0000 UTC m=+0.110666977 container died 88d5c4c639cbe3babfdb773f8894b81326c7aa6ad766a2cccf802c40e0ea68c8 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-01T02:03:22.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:21 smithi116 ceph-mon[25082]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-06-01T02:03:22.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:21 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/3760274103' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1076742300"}]': finished 2024-06-01T02:03:22.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:21 smithi116 ceph-mon[25082]: osdmap e111: 8 total, 8 up, 8 in 2024-06-01T02:03:22.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:21 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/3413543143' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/2654845353"}]: dispatch 2024-06-01T02:03:22.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:21 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:03:22.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:21 smithi116 ceph-mon[29958]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-06-01T02:03:22.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:21 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/3760274103' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/1076742300"}]': finished 2024-06-01T02:03:22.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:21 smithi116 ceph-mon[29958]: osdmap e111: 8 total, 8 up, 8 in 2024-06-01T02:03:22.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:21 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/3413543143' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/2654845353"}]: dispatch 2024-06-01T02:03:22.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:21 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:03:22.429 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:22 smithi161 podman[62245]: 2024-06-01 02:03:22.17420167 +0000 UTC m=+0.491357179 container cleanup 88d5c4c639cbe3babfdb773f8894b81326c7aa6ad766a2cccf802c40e0ea68c8 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-01T02:03:22.429 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:22 smithi161 bash[62245]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a 2024-06-01T02:03:22.429 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:22 smithi161 bash[62273]: Error: no container with name or ID "ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus.a" found: no such container 2024-06-01T02:03:22.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:22 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/3413543143' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/2654845353"}]': finished 2024-06-01T02:03:22.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:22 smithi161 ceph-mon[26055]: osdmap e112: 8 total, 8 up, 8 in 2024-06-01T02:03:22.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:22 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1574820012' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/314195764"}]: dispatch 2024-06-01T02:03:22.695 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:22 smithi161 podman[62257]: 2024-06-01 02:03:22.433819515 +0000 UTC m=+0.636657633 container remove 88d5c4c639cbe3babfdb773f8894b81326c7aa6ad766a2cccf802c40e0ea68c8 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-01T02:03:22.695 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:22 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-06-01T02:03:22.695 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:22 smithi161 systemd[1]: Stopped Ceph prometheus.a for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:03:22.695 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:22 smithi161 systemd[1]: Starting Ceph prometheus.a for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:03:22.695 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:22 smithi161 podman[62336]: 2024-06-01 02:03:22.571572952 +0000 UTC m=+0.018319521 image pull 514e6a882f6e74806a5856468489eeff8d7106095557578da96935e4d0ba4d9d quay.io/prometheus/prometheus:v2.33.4 2024-06-01T02:03:23.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:22 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/3413543143' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/2654845353"}]': finished 2024-06-01T02:03:23.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:22 smithi116 ceph-mon[29958]: osdmap e112: 8 total, 8 up, 8 in 2024-06-01T02:03:23.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:22 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1574820012' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/314195764"}]: dispatch 2024-06-01T02:03:23.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:22 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/3413543143' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:6801/2654845353"}]': finished 2024-06-01T02:03:23.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:22 smithi116 ceph-mon[25082]: osdmap e112: 8 total, 8 up, 8 in 2024-06-01T02:03:23.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:22 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1574820012' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/314195764"}]: dispatch 2024-06-01T02:03:23.077 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:22 smithi161 podman[62336]: 2024-06-01 02:03:22.745906618 +0000 UTC m=+0.192653160 container create e977f6ca1d73ad506f7af13b6f248cf5e8a761c803c155b46dfce087f79bfa01 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-01T02:03:23.077 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:22 smithi161 podman[62336]: 2024-06-01 02:03:22.956085645 +0000 UTC m=+0.402832194 container init e977f6ca1d73ad506f7af13b6f248cf5e8a761c803c155b46dfce087f79bfa01 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-01T02:03:23.077 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:22 smithi161 podman[62336]: 2024-06-01 02:03:22.958944529 +0000 UTC m=+0.405691073 container start e977f6ca1d73ad506f7af13b6f248cf5e8a761c803c155b46dfce087f79bfa01 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-01T02:03:23.077 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:22 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:03:22.982Z caller=main.go:475 level=info msg="No time or size retention was set so using the default time retention" duration=15d 2024-06-01T02:03:23.077 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:22 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:03:22.982Z caller=main.go:512 level=info msg="Starting Prometheus" version="(version=2.33.4, branch=HEAD, revision=83032011a5d3e6102624fe58241a374a7201fee8)" 2024-06-01T02:03:23.077 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:22 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:03:22.982Z caller=main.go:517 level=info build_context="(go=go1.17.7, user=root@d13bf69e7be8, date=20220222-16:51:28)" 2024-06-01T02:03:23.077 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:22 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:03:22.982Z caller=main.go:518 level=info host_details="(Linux 5.14.0-452.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024 x86_64 smithi161 (none))" 2024-06-01T02:03:23.077 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:22 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:03:22.982Z caller=main.go:519 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-06-01T02:03:23.077 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:22 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:03:22.982Z caller=main.go:520 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-06-01T02:03:23.077 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:22 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:03:22.984Z caller=web.go:570 level=info component=web msg="Start listening for connections" address=:9095 2024-06-01T02:03:23.078 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:22 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:03:22.985Z caller=main.go:923 level=info msg="Starting TSDB ..." 2024-06-01T02:03:23.078 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:22 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:03:22.986Z caller=tls_config.go:195 level=info component=web msg="TLS is disabled." http2=false 2024-06-01T02:03:23.078 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:22 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:03:22.988Z caller=head.go:493 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-06-01T02:03:23.078 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:22 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:03:22.989Z caller=head.go:527 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.260593ms 2024-06-01T02:03:23.078 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:22 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:03:22.989Z caller=head.go:533 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-06-01T02:03:23.078 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:22 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:03:22.991Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=4 2024-06-01T02:03:23.078 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:23 smithi161 bash[62336]: e977f6ca1d73ad506f7af13b6f248cf5e8a761c803c155b46dfce087f79bfa01 2024-06-01T02:03:23.406 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:23 smithi161 systemd[1]: Started Ceph prometheus.a for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:03:23.679 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:03:23.465Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=4 2024-06-01T02:03:23.679 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:03:23.659Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=2 maxSegment=4 2024-06-01T02:03:23.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:23 smithi161 ceph-mon[26055]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:03:23.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:23 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1574820012' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/314195764"}]': finished 2024-06-01T02:03:23.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:23 smithi161 ceph-mon[26055]: osdmap e113: 8 total, 8 up, 8 in 2024-06-01T02:03:23.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:23 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:23.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:23 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:23.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:23 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-01T02:03:23.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:23 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:23.945 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:03:23.700Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=3 maxSegment=4 2024-06-01T02:03:23.946 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:03:23.700Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=4 maxSegment=4 2024-06-01T02:03:23.946 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:03:23.700Z caller=head.go:610 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=41.342µs wal_replay_duration=711.100369ms total_replay_duration=712.419522ms 2024-06-01T02:03:23.946 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:03:23.705Z caller=main.go:944 level=info fs_type=XFS_SUPER_MAGIC 2024-06-01T02:03:23.946 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:03:23.705Z caller=main.go:947 level=info msg="TSDB started" 2024-06-01T02:03:23.946 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:03:23.705Z caller=main.go:1128 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-06-01T02:03:23.946 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:03:23.725Z caller=main.go:1165 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=19.599475ms db_storage=674ns remote_storage=1.393µs web_handler=724ns query_engine=700ns scrape=262.636µs scrape_sd=99.727µs notify=10.496µs notify_sd=9.508µs rules=18.880394ms 2024-06-01T02:03:23.946 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:03:23.725Z caller=main.go:896 level=info msg="Server is ready to receive web requests." 2024-06-01T02:03:23.992 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:23 smithi116 ceph-mon[29958]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:03:23.992 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:23 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1574820012' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/314195764"}]': finished 2024-06-01T02:03:23.992 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:23 smithi116 ceph-mon[29958]: osdmap e113: 8 total, 8 up, 8 in 2024-06-01T02:03:23.992 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:23 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:23.992 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:23 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:23.993 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:23 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-01T02:03:23.993 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:23 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:23.993 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:23 smithi116 ceph-mon[25082]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:03:23.993 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:23 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1574820012' entity='client.iscsi.foo.smithi116.gjuxzb' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.116:0/314195764"}]': finished 2024-06-01T02:03:23.993 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:23 smithi116 ceph-mon[25082]: osdmap e113: 8 total, 8 up, 8 in 2024-06-01T02:03:23.993 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:23 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:23.993 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:23 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:23.993 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:23 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-01T02:03:23.993 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:23 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:24.827 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:24 smithi161 ceph-mon[26055]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-01T02:03:25.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:24 smithi116 ceph-mon[25082]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-01T02:03:25.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:24 smithi116 ceph-mon[29958]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-01T02:03:25.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:25 smithi161 ceph-mon[26055]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-06-01T02:03:25.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:25 smithi161 ceph-mon[26055]: Upgrade: Updating mgr.y 2024-06-01T02:03:25.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:25 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:03:25.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:25 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:03:25.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:25 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-01T02:03:25.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:25 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:25.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:25 smithi161 ceph-mon[26055]: Deploying daemon mgr.y on smithi116 2024-06-01T02:03:25.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:25 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:25.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:25 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:25.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:25 smithi161 ceph-mon[26055]: pgmap v30: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:03:25.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:25 smithi161 ceph-mon[26055]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:03:25.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:25 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:26.009 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:25 smithi116 ceph-mon[25082]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-06-01T02:03:26.009 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:25 smithi116 ceph-mon[25082]: Upgrade: Updating mgr.y 2024-06-01T02:03:26.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:25 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:03:26.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:25 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:03:26.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:25 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-01T02:03:26.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:25 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:26.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:25 smithi116 ceph-mon[25082]: Deploying daemon mgr.y on smithi116 2024-06-01T02:03:26.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:25 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:26.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:25 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:26.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:25 smithi116 ceph-mon[25082]: pgmap v30: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:03:26.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:25 smithi116 ceph-mon[25082]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:03:26.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:25 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:26.011 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:25 smithi116 ceph-mon[29958]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-06-01T02:03:26.011 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:25 smithi116 ceph-mon[29958]: Upgrade: Updating mgr.y 2024-06-01T02:03:26.011 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:25 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:03:26.011 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:25 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:03:26.011 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:25 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-01T02:03:26.011 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:25 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:26.011 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:25 smithi116 ceph-mon[29958]: Deploying daemon mgr.y on smithi116 2024-06-01T02:03:26.011 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:25 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:26.011 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:25 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:26.011 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:25 smithi116 ceph-mon[29958]: pgmap v30: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:03:26.011 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:25 smithi116 ceph-mon[29958]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:03:26.012 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:25 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:26.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:26 smithi161 ceph-mon[26055]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:03:26.987 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:26 smithi116 ceph-mon[25082]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:03:26.987 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:26 smithi116 systemd[1]: Stopping Ceph mgr.y for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:03:26.988 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:26 smithi116 ceph-mon[29958]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:03:27.274 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:26 smithi116 podman[73320]: 2024-06-01 02:03:26.991188763 +0000 UTC m=+0.135275004 container died 0af4df0af300650d2dfc40f7cbc8869d89a260b43cb9c6a325dd0032b417b9a0 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-3d89acc) 2024-06-01T02:03:27.543 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:27 smithi116 podman[73320]: 2024-06-01 02:03:27.36058525 +0000 UTC m=+0.504671483 container cleanup 0af4df0af300650d2dfc40f7cbc8869d89a260b43cb9c6a325dd0032b417b9a0 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-06-01T02:03:27.543 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:27 smithi116 bash[73320]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y 2024-06-01T02:03:27.890 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:27 smithi116 ceph-mon[29958]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 967 B/s rd, 0 op/s 2024-06-01T02:03:27.890 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:27 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:27.890 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:27 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:27.890 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:27 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:27.891 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:27 smithi116 ceph-mon[25082]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 967 B/s rd, 0 op/s 2024-06-01T02:03:27.891 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:27 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:27.891 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:27 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:27.891 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:27 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:27.891 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:27 smithi116 podman[73332]: 2024-06-01 02:03:27.637033634 +0000 UTC m=+0.641154954 container remove 0af4df0af300650d2dfc40f7cbc8869d89a260b43cb9c6a325dd0032b417b9a0 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-06-01T02:03:27.891 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:27 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-06-01T02:03:27.891 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:27 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-06-01T02:03:27.891 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:27 smithi116 systemd[1]: Stopped Ceph mgr.y for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:03:27.891 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:27 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mgr.y.service: Consumed 24.019s CPU time. 2024-06-01T02:03:27.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:27 smithi161 ceph-mon[26055]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 99 MiB used, 715 GiB / 715 GiB avail; 967 B/s rd, 0 op/s 2024-06-01T02:03:27.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:27 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:27.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:27 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:27.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:27 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:28.200 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:27 smithi116 systemd[1]: Starting Ceph mgr.y for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:03:28.487 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:28 smithi116 podman[73460]: 2024-06-01 02:03:28.108006131 +0000 UTC m=+0.016873996 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:03:28.487 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:28 smithi116 podman[73460]: 2024-06-01 02:03:28.213058925 +0000 UTC m=+0.121926818 container create a6cc56fad4e6a39612392508910924fed8925406f4dddb517e34125c6c9b1e9e (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc) 2024-06-01T02:03:28.487 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:28 smithi116 podman[73460]: 2024-06-01 02:03:28.491220353 +0000 UTC m=+0.400088193 container init a6cc56fad4e6a39612392508910924fed8925406f4dddb517e34125c6c9b1e9e (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-06-01T02:03:28.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:28 smithi116 ceph-mon[25082]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1021 B/s rd, 0 op/s 2024-06-01T02:03:28.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:28 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:28.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:28 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:28.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:28 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:28.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:28 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:28.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:28 smithi116 ceph-mon[29958]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1021 B/s rd, 0 op/s 2024-06-01T02:03:28.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:28 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:28.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:28 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:28.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:28 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:28.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:28 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:28.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:28 smithi116 podman[73460]: 2024-06-01 02:03:28.494199527 +0000 UTC m=+0.403067375 container start a6cc56fad4e6a39612392508910924fed8925406f4dddb517e34125c6c9b1e9e (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, ceph=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-06-01T02:03:28.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:28 smithi116 ceph-mgr[73480]: -- 172.21.15.116:0/485064604 <== mon.0 v2:172.21.15.116:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f5e44e45a0 con 0x55f5e44b8c00 2024-06-01T02:03:28.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:28 smithi116 bash[73460]: a6cc56fad4e6a39612392508910924fed8925406f4dddb517e34125c6c9b1e9e 2024-06-01T02:03:28.795 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:28 smithi116 systemd[1]: Started Ceph mgr.y for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:03:28.795 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:28 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[73476]: 2024-06-01T02:03:28.650+0000 7fc962e0b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-01T02:03:28.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:28 smithi161 ceph-mon[26055]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1021 B/s rd, 0 op/s 2024-06-01T02:03:28.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:28 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:28.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:28 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:28.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:28 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:28.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:28 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:29.275 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:28 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[73476]: 2024-06-01T02:03:28.841+0000 7fc962e0b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-01T02:03:29.275 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:28 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[73476]: 2024-06-01T02:03:28.953+0000 7fc962e0b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-01T02:03:29.275 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[73476]: 2024-06-01T02:03:29.278+0000 7fc962e0b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-01T02:03:29.543 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[73476]: 2024-06-01T02:03:29.366+0000 7fc962e0b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-01T02:03:30.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 851 B/s rd, 0 op/s 2024-06-01T02:03:30.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:30.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:30.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:30.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:30.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: Upgrade: Setting container_image for all mgr 2024-06-01T02:03:30.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:30.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-06-01T02:03:30.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-06-01T02:03:30.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]': finished 2024-06-01T02:03:30.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-06-01T02:03:30.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-06-01T02:03:30.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]': finished 2024-06-01T02:03:30.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:30.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:30.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:03:30.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-01T02:03:30.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:30.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:30.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:30.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:30.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:30.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:30.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:03:30.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 2 op/s 2024-06-01T02:03:30.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:30.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:30.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:30.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:30.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:30.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:30.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:30.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:30.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:30.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:30.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:30.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:30.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:30.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:30.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:30.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:30.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:30.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:30.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:30.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:30.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:30.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:30.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:30.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:30.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:30.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:30.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:30.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 851 B/s rd, 0 op/s 2024-06-01T02:03:30.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:30.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:30.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:30.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:30.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: Upgrade: Setting container_image for all mgr 2024-06-01T02:03:30.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:30.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-06-01T02:03:30.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-06-01T02:03:30.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]': finished 2024-06-01T02:03:30.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-06-01T02:03:30.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-06-01T02:03:30.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]': finished 2024-06-01T02:03:30.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:30.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:30.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:03:30.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-01T02:03:30.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:30.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:30.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:30.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:30.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:30.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:30.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:03:30.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 2 op/s 2024-06-01T02:03:30.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:30.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:30.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:30.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:30.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:30.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:30.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:30.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:30.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:30.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:30.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:30.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:30.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:30.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:30.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:30.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:30.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:30.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:30.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:30.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:30.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:30.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:30.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:30.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:30.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:30.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:30.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:29 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 851 B/s rd, 0 op/s 2024-06-01T02:03:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: Upgrade: Setting container_image for all mgr 2024-06-01T02:03:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-06-01T02:03:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-06-01T02:03:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]': finished 2024-06-01T02:03:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-06-01T02:03:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-06-01T02:03:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]': finished 2024-06-01T02:03:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:03:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-01T02:03:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:03:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 2 op/s 2024-06-01T02:03:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:30.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:30.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:30.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:30.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:30.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:30.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:30.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:30.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:30.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:30.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:30.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:30.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:30.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:30.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:30.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:30.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:30.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:30.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:30.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:30.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:30.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:30.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:30.200 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:30.200 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:29 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:30.709 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[73476]: 2024-06-01T02:03:30.643+0000 7fc962e0b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-01T02:03:31.003 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[73476]: 2024-06-01T02:03:30.826+0000 7fc962e0b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-01T02:03:31.003 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.003 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: pgmap v40: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.004 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.004 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: pgmap v41: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.004 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: pgmap v42: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.004 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.004 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: pgmap v43: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.004 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: pgmap v44: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.004 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.004 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: pgmap v45: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.004 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: pgmap v46: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.004 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.004 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: pgmap v47: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.004 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: pgmap v48: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.004 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: pgmap v49: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: pgmap v50: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: pgmap v51: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.006 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: pgmap v52: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.006 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.006 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.006 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.006 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: pgmap v40: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: pgmap v41: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: pgmap v42: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: pgmap v43: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: pgmap v44: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: pgmap v45: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: pgmap v46: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: pgmap v47: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: pgmap v48: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: pgmap v49: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: pgmap v50: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: pgmap v51: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: pgmap v52: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: pgmap v53: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: pgmap v54: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.013 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.013 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.013 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.013 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.013 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.013 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.013 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.013 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.013 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.014 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.014 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.014 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.014 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: pgmap v53: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.014 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: pgmap v54: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.014 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.014 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.014 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.014 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.014 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.014 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.014 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.014 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.015 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.015 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.015 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.015 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.015 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.015 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.015 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.015 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.015 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.015 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.015 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.015 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.015 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.015 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.016 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.016 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.016 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.016 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.016 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.016 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.016 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.016 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.016 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.016 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:30 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: pgmap v40: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: pgmap v41: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: pgmap v42: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: pgmap v43: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: pgmap v44: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: pgmap v45: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: pgmap v46: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: pgmap v47: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: pgmap v48: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: pgmap v49: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: pgmap v50: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: pgmap v51: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: pgmap v52: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: pgmap v53: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: pgmap v54: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.200 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.200 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.200 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.200 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.200 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.200 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:30 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.255 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[73476]: 2024-06-01T02:03:31.006+0000 7fc962e0b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-01T02:03:31.543 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[73476]: 2024-06-01T02:03:31.324+0000 7fc962e0b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-01T02:03:31.543 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[73476]: 2024-06-01T02:03:31.445+0000 7fc962e0b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-01T02:03:31.904 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[73476]: 2024-06-01T02:03:31.666+0000 7fc962e0b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-01T02:03:31.904 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: pgmap v55: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.904 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: pgmap v56: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.904 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.904 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: pgmap v57: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.905 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: pgmap v58: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.905 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.905 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: pgmap v59: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.905 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: pgmap v60: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.905 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.905 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: pgmap v61: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.905 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: pgmap v62: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.905 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.905 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: pgmap v63: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.905 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: pgmap v64: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.905 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.905 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: pgmap v65: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.905 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: pgmap v66: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.905 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.906 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.906 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.906 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.906 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.906 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: pgmap v67: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.906 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: pgmap v68: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.906 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.906 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.906 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.906 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.906 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.906 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.906 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: pgmap v69: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.907 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: pgmap v55: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.907 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: pgmap v56: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.907 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.907 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: pgmap v57: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.907 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: pgmap v58: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.907 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.907 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: pgmap v59: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.907 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: pgmap v60: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.907 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.908 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: pgmap v61: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.908 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: pgmap v62: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.908 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.908 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: pgmap v63: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.908 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: pgmap v64: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.908 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.908 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: pgmap v65: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.908 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: pgmap v66: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.908 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.908 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.908 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.908 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.908 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.909 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: pgmap v67: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.909 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: pgmap v68: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.909 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.909 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.909 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.909 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.909 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.909 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.909 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: pgmap v69: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.909 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: pgmap v70: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.909 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.910 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.910 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.910 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.910 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.910 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.910 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: pgmap v71: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.910 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: pgmap v72: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.910 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.910 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.910 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.910 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.910 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.911 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.911 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: pgmap v73: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.911 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: pgmap v74: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.911 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.911 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.911 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.911 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.911 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.911 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.911 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.911 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.911 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.911 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.911 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.912 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.912 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.912 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: pgmap v70: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.912 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.912 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.912 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.912 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.912 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.912 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.912 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: pgmap v71: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.912 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: pgmap v72: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.913 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.913 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.913 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.913 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.913 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.913 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.913 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: pgmap v73: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.913 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: pgmap v74: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.913 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.913 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.913 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.913 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.913 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.913 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.914 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.914 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.914 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.914 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.914 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.914 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.914 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.914 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.914 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.914 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.914 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.915 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.915 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.915 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.915 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.915 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.915 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.915 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.915 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.915 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.915 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.915 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.915 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.916 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.916 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.916 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.916 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.916 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.916 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.916 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.916 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.916 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.916 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.916 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.916 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: pgmap v55: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: pgmap v56: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: pgmap v57: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: pgmap v58: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: pgmap v59: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: pgmap v60: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: pgmap v61: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: pgmap v62: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: pgmap v63: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: pgmap v64: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: pgmap v65: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: pgmap v66: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: pgmap v67: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: pgmap v68: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: pgmap v69: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: pgmap v70: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: pgmap v71: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: pgmap v72: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: pgmap v73: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: pgmap v74: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:31.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:31.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:31.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:31.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:31.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:31.951 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:31.951 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:31 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:32.211 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:31 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[73476]: 2024-06-01T02:03:31.907+0000 7fc962e0b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-01T02:03:32.757 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: pgmap v75: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.757 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: pgmap v76: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.757 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:32.757 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: pgmap v77: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.757 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: pgmap v78: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.758 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:32.758 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: pgmap v79: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.758 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: pgmap v80: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.758 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:32.758 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: pgmap v81: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.758 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: pgmap v82: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.758 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:32.758 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: pgmap v83: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.758 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: pgmap v84: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.758 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:32.758 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:32.758 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:32.758 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:32.758 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:32.759 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: pgmap v85: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.759 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: pgmap v86: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.759 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:32.759 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:32.759 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:32.759 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:32.759 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:32.759 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:32.759 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: pgmap v87: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.759 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: pgmap v88: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.759 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:32.759 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:32.759 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:32.760 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:32.760 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:32.760 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:32.760 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: pgmap v89: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.760 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: pgmap v90: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.760 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:32.760 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:32.760 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:32.760 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:32.760 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:32.760 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:32.760 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:32.760 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:32.761 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:32.761 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:32.761 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:32.761 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:32.761 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:32.761 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:32.761 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:32.761 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:32.761 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:32.761 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:32.761 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:32.761 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:32.761 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:32.761 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:32.762 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:32.762 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:32.762 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:32 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:32.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[73476]: 2024-06-01T02:03:32.585+0000 7fc962e0b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-01T02:03:32.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: pgmap v75: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: pgmap v76: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:32.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: pgmap v77: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: pgmap v78: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:32.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: pgmap v79: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: pgmap v80: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:32.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: pgmap v81: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: pgmap v82: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:32.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: pgmap v83: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: pgmap v84: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:32.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:32.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:32.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:32.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:32.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: pgmap v85: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: pgmap v86: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:32.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:32.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:32.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:32.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:32.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:32.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: pgmap v87: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: pgmap v88: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:32.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:32.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: pgmap v75: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: pgmap v76: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:32.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: pgmap v77: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: pgmap v78: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:32.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: pgmap v79: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: pgmap v80: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:32.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: pgmap v81: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: pgmap v82: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:32.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: pgmap v83: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: pgmap v84: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:32.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:32.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:32.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:32.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:32.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: pgmap v85: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: pgmap v86: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:32.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:32.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:32.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:32.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:32.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:32.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: pgmap v87: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: pgmap v88: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:32.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:32.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:32.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:32.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:32.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:32.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: pgmap v89: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: pgmap v90: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:32.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:32.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:32.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:32.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:32.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:32.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:32.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:32.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:32.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:32.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:32.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:32.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:32.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:32.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:32.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:32.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:32.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:32.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:32.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:32.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:32.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:32.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:32.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:32.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:32.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:32.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:32.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: pgmap v89: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: pgmap v90: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:32.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:32.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:32.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:32.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:32.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:32.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:32.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:32.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:32.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:32.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:32.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:32.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:32.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:32.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:32.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:32.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:32.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:32.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:32.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:32.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:32.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:32.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:32.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:32.804 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:32.804 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:32.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:32.805 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:32 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:33.150 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[73476]: 2024-06-01T02:03:33.050+0000 7fc962e0b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-01T02:03:33.151 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[73476]: 2024-06-01T02:03:33.154+0000 7fc962e0b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-01T02:03:33.502 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[73476]: 2024-06-01T02:03:33.369+0000 7fc962e0b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-01T02:03:33.537 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:03:33.504Z caller=manager.go:609 level=warn component="rule manager" group=nodes msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on(instance) group_left(nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi116\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:03:33.764 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: pgmap v91: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:33.765 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: pgmap v92: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:33.765 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:33.765 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: pgmap v93: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 11 op/s 2024-06-01T02:03:33.765 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: pgmap v94: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 11 op/s 2024-06-01T02:03:33.765 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:33.765 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: pgmap v95: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 12 KiB/s rd, 11 op/s 2024-06-01T02:03:33.765 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: pgmap v96: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 12 KiB/s rd, 11 op/s 2024-06-01T02:03:33.765 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:33.765 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: pgmap v97: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 12 KiB/s rd, 11 op/s 2024-06-01T02:03:33.765 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: pgmap v98: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 12 KiB/s rd, 11 op/s 2024-06-01T02:03:33.765 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:33.765 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:33.766 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: pgmap v99: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:33.766 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: pgmap v100: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:33.766 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:33.766 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:33.766 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:33.766 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:33.766 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:33.766 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:33.766 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: pgmap v101: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:33.766 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: pgmap v102: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:33.766 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:33.766 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:33.766 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:33.766 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:33.767 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:33.767 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:33.767 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: pgmap v103: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:33.767 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: pgmap v104: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:33.767 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:33.767 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:33.767 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:33.768 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:33.768 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:33.768 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:33.768 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: pgmap v105: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:33.768 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: pgmap v106: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:33.768 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:33.768 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:33.768 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:33.768 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:33.768 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:33.768 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:33.768 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:33.768 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:33.768 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:33.769 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:33.769 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:33.769 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:33.769 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:33.769 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:33.769 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:33.769 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:33.769 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:33.769 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:33.769 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:33.769 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:33.770 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: pgmap v91: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:33.770 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: pgmap v92: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:33.770 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:33.770 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: pgmap v93: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 11 op/s 2024-06-01T02:03:33.770 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: pgmap v94: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 11 op/s 2024-06-01T02:03:33.770 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:33.770 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: pgmap v95: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 12 KiB/s rd, 11 op/s 2024-06-01T02:03:33.770 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: pgmap v96: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 12 KiB/s rd, 11 op/s 2024-06-01T02:03:33.770 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:33.770 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: pgmap v97: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 12 KiB/s rd, 11 op/s 2024-06-01T02:03:33.770 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: pgmap v98: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 12 KiB/s rd, 11 op/s 2024-06-01T02:03:33.770 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:33.771 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:33.771 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: pgmap v99: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:33.771 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: pgmap v100: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:33.771 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:33.771 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:33.771 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:33.771 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:33.771 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:33.771 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:33.771 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: pgmap v101: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:33.771 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: pgmap v102: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:33.771 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:33.771 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:33.771 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:33.772 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:33.772 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:33.772 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:33.772 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: pgmap v103: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:33.772 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:33.772 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:33.772 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:33.772 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:33.772 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:33.772 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:33.772 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:33.773 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:33.773 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:33.773 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:33.773 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:33.773 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:33.773 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: pgmap v104: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:33.773 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:33.773 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:33.773 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:33.773 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:33.774 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:33.774 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:33.774 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: pgmap v105: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:33.774 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: pgmap v106: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:33.774 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:33.774 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:33.774 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:33.774 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:33.774 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:33.774 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:33.774 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:33.774 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:33.774 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:33.775 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:33.775 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:33.775 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:33.775 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:33.775 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:33.775 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:33.775 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:33.775 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:33.775 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:33.775 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:33.775 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:33.775 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:33.775 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:33.775 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:33.775 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:33.776 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:33.776 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:33.776 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:33.776 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:33.776 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:33.776 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:33.776 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:33.776 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:33.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: pgmap v91: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:33.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: pgmap v92: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:33.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:33.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: pgmap v93: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 11 op/s 2024-06-01T02:03:33.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: pgmap v94: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 11 op/s 2024-06-01T02:03:33.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:33.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: pgmap v95: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 12 KiB/s rd, 11 op/s 2024-06-01T02:03:33.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: pgmap v96: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 12 KiB/s rd, 11 op/s 2024-06-01T02:03:33.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:33.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: pgmap v97: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 12 KiB/s rd, 11 op/s 2024-06-01T02:03:33.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: pgmap v98: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 12 KiB/s rd, 11 op/s 2024-06-01T02:03:33.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:33.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:33.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: pgmap v99: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:33.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: pgmap v100: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:33.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:33.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:33.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:33.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:33.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:33.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:33.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: pgmap v101: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:33.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: pgmap v102: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:33.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:33.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:33.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:33.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:33.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:33.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:33.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: pgmap v103: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:33.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: pgmap v104: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:33.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:33.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:33.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:33.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:33.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:33.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:33.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: pgmap v105: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:33.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: pgmap v106: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:33.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:33.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:33.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:33.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:33.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:33.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:33.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:33.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:33.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:33.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:33.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:33.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:33.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:33.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:33.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:33.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:33.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:33.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:33.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:33.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:33.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:33.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:33.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:33.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:33.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:33.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:33.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:33.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:33.951 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:33.951 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:33.951 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:33.951 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:33 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:34.034 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:33 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[73476]: 2024-06-01T02:03:33.938+0000 7fc962e0b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-01T02:03:34.293 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[73476]: 2024-06-01T02:03:34.038+0000 7fc962e0b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-01T02:03:34.656 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[73476]: 2024-06-01T02:03:34.303+0000 7fc962e0b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-01T02:03:34.657 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[73476]: 2024-06-01T02:03:34.449+0000 7fc962e0b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-01T02:03:34.657 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[73476]: 2024-06-01T02:03:34.551+0000 7fc962e0b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-01T02:03:34.962 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[73476]: 2024-06-01T02:03:34.676+0000 7fc962e0b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-01T02:03:34.962 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: pgmap v107: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:34.962 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: pgmap v108: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:34.962 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:34.963 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: pgmap v109: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:34.963 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: pgmap v110: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:34.963 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:34.963 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: pgmap v111: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:34.963 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: pgmap v112: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:34.963 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:34.963 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: pgmap v113: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:34.963 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: pgmap v114: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:34.963 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:34.963 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: pgmap v115: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:34.963 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: pgmap v116: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:34.963 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:34.963 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: pgmap v117: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:34.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: pgmap v118: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:34.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:34.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:34.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:34.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:34.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:34.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: pgmap v119: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:34.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: pgmap v120: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:34.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:34.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:34.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:34.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:34.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:34.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:34.965 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: pgmap v121: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:34.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: pgmap v107: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:34.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: pgmap v108: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:34.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:34.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: pgmap v109: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:34.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: pgmap v110: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:34.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:34.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: pgmap v111: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:34.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: pgmap v112: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:34.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:34.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: pgmap v113: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:34.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: pgmap v114: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:34.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:34.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: pgmap v115: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:34.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: pgmap v116: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:34.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:34.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: pgmap v117: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:34.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: pgmap v118: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:34.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:34.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:34.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:34.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:34.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:34.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: pgmap v119: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:34.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: pgmap v120: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:34.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:34.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:34.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:34.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:34.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:34.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:34.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: pgmap v121: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:34.967 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: pgmap v122: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:34.967 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:34.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:34.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:34.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:34.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:34.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:34.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:34.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:34.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:34.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:34.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:34.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:34.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:34.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:34.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:34.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:34.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:34.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:34.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:34.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:34.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:34.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:34.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:34.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:34.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:34.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:34.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:34.970 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:34.970 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:34.970 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:34.970 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:34.970 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:34.970 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:34.970 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:34.970 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:34.970 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:34.970 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:34.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: pgmap v122: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:34.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:34.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:34.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:34.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:34.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:34.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:34.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:34.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:34.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:34.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:34.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:34.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:34.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:34.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:34.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:34.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:34.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:34.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:34.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:34.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:34.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:34.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:34.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:34.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:34.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:34.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:34.973 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:34.973 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:34.973 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:34.973 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:34.973 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:34.973 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:34.973 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:34.973 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:34.973 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:34.973 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:34.973 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:34 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:35.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: pgmap v107: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: pgmap v108: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:35.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: pgmap v109: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: pgmap v110: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:35.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: pgmap v111: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: pgmap v112: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:35.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: pgmap v113: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: pgmap v114: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:35.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: pgmap v115: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: pgmap v116: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:35.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: pgmap v117: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: pgmap v118: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:35.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:35.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:35.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:35.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:35.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: pgmap v119: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: pgmap v120: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:35.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:35.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:35.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:35.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:35.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:35.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: pgmap v121: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: pgmap v122: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:35.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:35.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:35.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:35.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:35.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:35.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:35.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:35.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:35.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:35.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:35.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:35.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:35.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:35.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:35.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:35.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:35.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:35.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:35.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:35.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:35.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:35.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:35.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:35.200 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:35.200 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:35.200 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:35.200 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:35.200 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:35.200 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:35.200 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:35.200 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:35.200 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:35.200 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:35.200 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:35.200 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:35.200 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:34 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:35.249 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[73476]: 2024-06-01T02:03:34.999+0000 7fc962e0b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-01T02:03:35.812 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[73476]: 2024-06-01T02:03:35.661+0000 7fc962e0b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-01T02:03:35.813 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: pgmap v123: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.813 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: pgmap v124: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.813 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:35.813 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: pgmap v125: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.813 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: pgmap v126: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.813 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:35.813 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: pgmap v127: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.813 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: pgmap v128: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.813 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:35.813 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: pgmap v129: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.814 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: pgmap v130: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.814 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:35.814 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: pgmap v131: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.814 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: pgmap v132: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.814 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:35.814 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: pgmap v133: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.814 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: pgmap v134: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.814 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:35.814 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: pgmap v135: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.814 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: pgmap v136: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.814 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:35.814 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:35.814 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:35.815 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:35.815 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:35.815 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: pgmap v137: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.815 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: pgmap v138: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.815 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:35.815 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:35.815 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:35.815 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:35.815 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: pgmap v123: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.815 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: pgmap v124: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.815 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:35.816 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: pgmap v125: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.816 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: pgmap v126: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.816 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:35.816 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: pgmap v127: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.816 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: pgmap v128: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.816 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:35.816 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: pgmap v129: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.816 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: pgmap v130: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.816 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:35.816 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: pgmap v131: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.816 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: pgmap v132: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.816 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:35.816 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: pgmap v133: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.816 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: pgmap v134: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.817 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:35.817 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: pgmap v135: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.817 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: pgmap v136: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.817 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:35.817 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:35.817 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:35.817 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:35.817 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:35.817 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: pgmap v137: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.817 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: pgmap v138: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.817 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:35.817 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:35.817 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:35.817 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:35.818 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:35.818 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:35.818 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: pgmap v139: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.818 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: pgmap v140: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.818 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:35.818 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:35.818 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:35.818 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:35.819 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:35.819 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:35.819 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:35.819 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:35.819 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:35.819 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:35.819 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:35.819 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:35.819 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:35.819 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:35.819 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:35.819 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:35.819 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:35.819 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:35.820 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:35.820 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:35.820 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:35.820 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:35.820 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:35.820 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:35.820 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:35.820 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:35.820 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:35.820 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:35.820 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:35.821 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: pgmap v139: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.821 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: pgmap v140: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:35.821 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:35.821 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:35.821 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:35.821 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:35.821 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:35.821 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:35.821 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:35.821 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:35.821 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:35.821 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:35.821 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:35.822 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:35.822 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:35.822 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:35.822 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:35.822 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:35.822 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:35.822 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:35.822 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:35.822 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:35.822 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:35.822 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:35.822 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:35.822 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:35.823 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:35.823 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:35.823 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: pgmap v123: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: pgmap v124: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:36.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: pgmap v125: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: pgmap v126: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:36.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: pgmap v127: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: pgmap v128: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:36.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: pgmap v129: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: pgmap v130: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:36.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: pgmap v131: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: pgmap v132: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:36.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: pgmap v133: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: pgmap v134: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:36.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: pgmap v135: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: pgmap v136: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:36.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:36.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:36.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:36.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:36.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: pgmap v137: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: pgmap v138: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:36.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:36.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:36.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:36.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:36.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: pgmap v139: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: pgmap v140: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.097 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:36.097 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:36.097 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:36.097 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:36.097 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:36.097 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.156 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:36.156 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:36.156 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:36.156 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:36.156 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.156 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:36.157 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:36.157 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:36.157 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:36.157 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.157 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:36.157 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:36.157 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:36.157 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:36.157 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.157 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:36.157 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:36.157 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:36.157 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:36.158 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:35 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.293 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[73476]: 2024-06-01T02:03:35.920+0000 7fc962e0b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-01T02:03:36.293 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:35 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[73476]: [01/Jun/2024:02:03:35] ENGINE Bus STARTING 2024-06-01T02:03:36.293 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[73476]: CherryPy Checker: 2024-06-01T02:03:36.293 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[73476]: The Application mounted at '' has an empty config. 2024-06-01T02:03:36.293 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[73476]: 2024-06-01T02:03:36.294 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[73476]: [01/Jun/2024:02:03:36] ENGINE Serving on http://:::9283 2024-06-01T02:03:36.294 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[73476]: [01/Jun/2024:02:03:36] ENGINE Bus STARTED 2024-06-01T02:03:36.944 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:03:36] "GET /metrics HTTP/1.1" 200 38249 "" "Prometheus/2.33.4" 2024-06-01T02:03:36.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: pgmap v141: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: pgmap v142: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:36.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: pgmap v143: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: pgmap v144: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:36.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: pgmap v145: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: pgmap v146: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:36.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: pgmap v147: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: pgmap v148: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:36.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: pgmap v149: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: pgmap v150: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:36.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:36.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:36.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:36.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:36.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: pgmap v151: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: pgmap v152: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:36.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: Standby manager daemon y restarted 2024-06-01T02:03:36.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: Standby manager daemon y started 2024-06-01T02:03:36.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.116:0/4036443679' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-01T02:03:36.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.116:0/4036443679' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T02:03:36.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.116:0/4036443679' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-01T02:03:36.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.116:0/4036443679' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T02:03:36.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:36.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:36.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:36.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:36.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: pgmap v153: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: pgmap v154: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:36.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:03:36.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:36.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:36.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:36.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:36.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:36.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:36.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:36.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:36.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:36.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:36.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:36.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:36.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:03:36.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:36.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:36.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:36.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:36.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:36.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:36.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:36.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:36.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:36 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: pgmap v141: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: pgmap v142: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:36.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: pgmap v143: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: pgmap v144: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:36.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: pgmap v145: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: pgmap v146: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:36.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: pgmap v147: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: pgmap v148: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:36.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: pgmap v149: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: pgmap v150: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:36.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:36.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:36.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:36.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:36.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: pgmap v151: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: pgmap v152: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:36.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: Standby manager daemon y restarted 2024-06-01T02:03:36.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: Standby manager daemon y started 2024-06-01T02:03:36.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.116:0/4036443679' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-01T02:03:36.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.116:0/4036443679' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T02:03:36.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.116:0/4036443679' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-01T02:03:36.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.116:0/4036443679' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T02:03:36.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:36.967 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: pgmap v141: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.967 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: pgmap v142: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.967 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:36.967 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: pgmap v143: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.967 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: pgmap v144: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.967 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:36.967 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: pgmap v145: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: pgmap v146: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:36.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: pgmap v147: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: pgmap v148: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:36.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: pgmap v149: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: pgmap v150: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:36.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:36.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:36.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:36.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:36.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: pgmap v151: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: pgmap v152: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:36.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: Standby manager daemon y restarted 2024-06-01T02:03:36.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: Standby manager daemon y started 2024-06-01T02:03:36.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.116:0/4036443679' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-01T02:03:36.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.116:0/4036443679' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T02:03:36.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.116:0/4036443679' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-01T02:03:36.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.116:0/4036443679' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T02:03:36.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:36.970 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:36.970 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:36.970 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:36.970 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: pgmap v153: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.970 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: pgmap v154: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.970 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.970 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:36.970 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:03:36.970 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:36.970 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:36.970 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:36.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:36.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:36.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:36.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:36.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:36.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:36.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:36.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:36.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:36.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:03:36.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:36.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:36.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:36.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:36.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:36.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:36.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:36.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:36.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.973 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:36.973 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:36.973 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:36.973 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: pgmap v153: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.973 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: pgmap v154: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:36.973 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.973 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:36.973 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:03:36.973 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:36.973 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:36.973 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:36.973 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:36.973 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.974 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:36.974 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:36.974 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:36.974 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:36.974 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.974 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:36.974 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:36.974 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:36.974 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:36.974 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.974 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:03:36.974 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:36.974 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:36.975 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:36.975 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:36.975 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.975 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.975 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.975 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:36.975 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:36.975 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:36.975 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:36.975 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:36.975 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:36 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:38.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[29958]: pgmap v155: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[29958]: pgmap v156: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:38.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[29958]: pgmap v157: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[29958]: pgmap v158: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:38.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[29958]: pgmap v159: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[29958]: pgmap v160: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:38.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[29958]: pgmap v161: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[29958]: pgmap v162: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:38.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[29958]: pgmap v163: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[29958]: pgmap v164: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[29958]: pgmap v165: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[29958]: pgmap v166: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:38.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:38.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:38.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[29958]: pgmap v167: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[29958]: pgmap v168: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:38.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:38.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[29958]: mgrmap e36: x(active, since 30s), standbys: y 2024-06-01T02:03:38.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[25082]: pgmap v155: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[25082]: pgmap v156: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:38.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[25082]: pgmap v157: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[25082]: pgmap v158: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:38.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[25082]: pgmap v159: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[25082]: pgmap v160: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:38.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[25082]: pgmap v161: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[25082]: pgmap v162: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:38.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[25082]: pgmap v163: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[25082]: pgmap v164: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[25082]: pgmap v165: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[25082]: pgmap v166: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:38.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:38.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:38.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[25082]: pgmap v167: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[25082]: pgmap v168: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:38.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:38.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:37 smithi116 ceph-mon[25082]: mgrmap e36: x(active, since 30s), standbys: y 2024-06-01T02:03:38.054 INFO:teuthology.orchestra.run.smithi116.stdout:true 2024-06-01T02:03:38.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:37 smithi161 ceph-mon[26055]: pgmap v155: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:37 smithi161 ceph-mon[26055]: pgmap v156: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:37 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:38.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:37 smithi161 ceph-mon[26055]: pgmap v157: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:37 smithi161 ceph-mon[26055]: pgmap v158: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:37 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:38.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:37 smithi161 ceph-mon[26055]: pgmap v159: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:37 smithi161 ceph-mon[26055]: pgmap v160: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:37 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:38.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:37 smithi161 ceph-mon[26055]: pgmap v161: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:37 smithi161 ceph-mon[26055]: pgmap v162: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:37 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:38.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:37 smithi161 ceph-mon[26055]: pgmap v163: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:37 smithi161 ceph-mon[26055]: pgmap v164: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:37 smithi161 ceph-mon[26055]: pgmap v165: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:37 smithi161 ceph-mon[26055]: pgmap v166: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:37 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:38.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:37 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:38.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:37 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:38.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:37 smithi161 ceph-mon[26055]: pgmap v167: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:37 smithi161 ceph-mon[26055]: pgmap v168: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:03:38.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:37 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:38.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:37 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:38.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:37 smithi161 ceph-mon[26055]: mgrmap e36: x(active, since 30s), standbys: y 2024-06-01T02:03:38.763 INFO:teuthology.orchestra.run.smithi116.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-01T02:03:38.763 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi116 smithi116 running 2s ago 12m - - 2024-06-01T02:03:38.763 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi161 smithi161 running 2s ago 12m - - 2024-06-01T02:03:38.763 INFO:teuthology.orchestra.run.smithi116.stdout:alertmanager.a smithi116 *:9093,9094 running (98s) 2s ago 6m 14.4M - 0.23.0 ba2b418f427c e0c297162ded 2024-06-01T02:03:38.764 INFO:teuthology.orchestra.run.smithi116.stdout:grafana.a smithi161 *:3000 running (75s) 2s ago 6m 46.0M - 8.3.5 dad864ee21e9 00b0a78ee88e 2024-06-01T02:03:38.764 INFO:teuthology.orchestra.run.smithi116.stdout:iscsi.foo.smithi116.gjuxzb smithi116 unknown 2s ago 5m - - 2024-06-01T02:03:38.764 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.x smithi161 *:8443,9283,8765 running (40s) 2s ago 11m 504M - 19.0.0-2946-g3d89accf b045ac545f82 2d7fbb85e5f7 2024-06-01T02:03:38.764 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.y smithi116 *:8443,9283,8765 running (69s) 2s ago 13m 450M - 19.0.0-2946-g3d89accf b045ac545f82 0af4df0af300 2024-06-01T02:03:38.764 INFO:teuthology.orchestra.run.smithi116.stdout:mon.a smithi116 running (13m) 2s ago 13m 63.0M 2048M 17.2.0 e1d6a67b021e c821092a538a 2024-06-01T02:03:38.764 INFO:teuthology.orchestra.run.smithi116.stdout:mon.b smithi161 running (11m) 2s ago 11m 58.0M 2048M 17.2.0 e1d6a67b021e 9011986aa6e3 2024-06-01T02:03:38.764 INFO:teuthology.orchestra.run.smithi116.stdout:mon.c smithi116 running (11m) 2s ago 11m 55.7M 2048M 17.2.0 e1d6a67b021e 4a6fe0b16d63 2024-06-01T02:03:38.764 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.a smithi116 *:9100 running (106s) 2s ago 7m 8334k - 1.3.1 1dbe0e931976 e347748d17bb 2024-06-01T02:03:38.764 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.b smithi161 *:9100 running (80s) 2s ago 7m 17.5M - 1.3.1 1dbe0e931976 6e66182f5a1a 2024-06-01T02:03:38.764 INFO:teuthology.orchestra.run.smithi116.stdout:osd.0 smithi116 running (11m) 2s ago 11m 54.2M 4096M 17.2.0 e1d6a67b021e 0cc929b0ecad 2024-06-01T02:03:38.764 INFO:teuthology.orchestra.run.smithi116.stdout:osd.1 smithi116 running (10m) 2s ago 10m 55.2M 4096M 17.2.0 e1d6a67b021e 6ff7fc5abe58 2024-06-01T02:03:38.764 INFO:teuthology.orchestra.run.smithi116.stdout:osd.2 smithi116 running (10m) 2s ago 10m 52.2M 4096M 17.2.0 e1d6a67b021e 6aecbb10256b 2024-06-01T02:03:38.764 INFO:teuthology.orchestra.run.smithi116.stdout:osd.3 smithi116 running (9m) 2s ago 9m 54.3M 4096M 17.2.0 e1d6a67b021e a802290f5290 2024-06-01T02:03:38.765 INFO:teuthology.orchestra.run.smithi116.stdout:osd.4 smithi161 running (9m) 2s ago 9m 56.7M 4096M 17.2.0 e1d6a67b021e 70b3f89b0bd9 2024-06-01T02:03:38.765 INFO:teuthology.orchestra.run.smithi116.stdout:osd.5 smithi161 running (9m) 2s ago 9m 51.2M 4096M 17.2.0 e1d6a67b021e 68358b78a25d 2024-06-01T02:03:38.765 INFO:teuthology.orchestra.run.smithi116.stdout:osd.6 smithi161 running (8m) 2s ago 8m 50.9M 4096M 17.2.0 e1d6a67b021e 50abeb0551e7 2024-06-01T02:03:38.765 INFO:teuthology.orchestra.run.smithi116.stdout:osd.7 smithi161 running (8m) 2s ago 8m 54.6M 4096M 17.2.0 e1d6a67b021e 6004e6983735 2024-06-01T02:03:38.765 INFO:teuthology.orchestra.run.smithi116.stdout:prometheus.a smithi161 *:9095 running (16s) 2s ago 7m 51.5M - 2.33.4 514e6a882f6e e977f6ca1d73 2024-06-01T02:03:38.765 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi116.vzjrxj smithi116 *:8000 running (5m) 2s ago 5m 74.8M - 17.2.0 e1d6a67b021e df4fe3c1dbc3 2024-06-01T02:03:38.765 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi161.ashqxz smithi161 *:8000 running (5m) 2s ago 5m 76.2M - 17.2.0 e1d6a67b021e 3eecc1f3e2d3 2024-06-01T02:03:38.765 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi116.hokbmf smithi116 *:80 running (5m) 2s ago 5m 75.3M - 17.2.0 e1d6a67b021e 10950ed75dbd 2024-06-01T02:03:38.765 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi161.axjlzw smithi161 *:80 running (5m) 2s ago 5m 74.5M - 17.2.0 e1d6a67b021e 0b5d5ce47772 2024-06-01T02:03:38.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:38 smithi161 ceph-mon[26055]: from='client.15570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:03:38.945 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:38 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:03:38.660Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"172.21.15.161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"smithi161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:03:38.951 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:38 smithi116 ceph-mon[29958]: from='client.15570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:03:38.951 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:38 smithi116 ceph-mon[25082]: from='client.15570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:03:39.212 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:03:39.212 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": { 2024-06-01T02:03:39.212 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 3 2024-06-01T02:03:39.212 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:03:39.212 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": { 2024-06-01T02:03:39.212 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-01T02:03:39.212 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:03:39.212 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": { 2024-06-01T02:03:39.212 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2024-06-01T02:03:39.212 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:03:39.212 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": {}, 2024-06-01T02:03:39.213 INFO:teuthology.orchestra.run.smithi116.stdout: "rgw": { 2024-06-01T02:03:39.213 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-06-01T02:03:39.213 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:03:39.213 INFO:teuthology.orchestra.run.smithi116.stdout: "overall": { 2024-06-01T02:03:39.213 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 15, 2024-06-01T02:03:39.213 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-01T02:03:39.213 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T02:03:39.213 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:03:39.600 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:03:39.600 INFO:teuthology.orchestra.run.smithi116.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-01T02:03:39.600 INFO:teuthology.orchestra.run.smithi116.stdout: "in_progress": true, 2024-06-01T02:03:39.601 INFO:teuthology.orchestra.run.smithi116.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-01T02:03:39.601 INFO:teuthology.orchestra.run.smithi116.stdout: "services_complete": [ 2024-06-01T02:03:39.601 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr" 2024-06-01T02:03:39.601 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-06-01T02:03:39.601 INFO:teuthology.orchestra.run.smithi116.stdout: "progress": "2/23 daemons upgraded", 2024-06-01T02:03:39.601 INFO:teuthology.orchestra.run.smithi116.stdout: "message": "Currently upgrading mgr daemons", 2024-06-01T02:03:39.601 INFO:teuthology.orchestra.run.smithi116.stdout: "is_paused": false 2024-06-01T02:03:39.601 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:03:39.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:39 smithi116 ceph-mon[29958]: from='client.25474 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:03:39.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:39 smithi116 ceph-mon[29958]: from='client.15582 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:03:39.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:39 smithi116 ceph-mon[29958]: pgmap v169: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 2 op/s 2024-06-01T02:03:39.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:39 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/2084232182' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:39.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:39 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:03:39.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:39 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:03:39.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:39 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-01T02:03:39.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:39 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:39.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:39 smithi116 ceph-mon[25082]: from='client.25474 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:03:39.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:39 smithi116 ceph-mon[25082]: from='client.15582 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:03:39.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:39 smithi116 ceph-mon[25082]: pgmap v169: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 2 op/s 2024-06-01T02:03:39.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:39 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/2084232182' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:39.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:39 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:03:39.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:39 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:03:39.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:39 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-01T02:03:39.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:39 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:40.010 INFO:teuthology.orchestra.run.smithi116.stdout:HEALTH_WARN 1 failed cephadm daemon(s) 2024-06-01T02:03:40.010 INFO:teuthology.orchestra.run.smithi116.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-06-01T02:03:40.010 INFO:teuthology.orchestra.run.smithi116.stdout: daemon iscsi.foo.smithi116.gjuxzb on smithi116 is in unknown state 2024-06-01T02:03:40.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:39 smithi161 ceph-mon[26055]: from='client.25474 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:03:40.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:39 smithi161 ceph-mon[26055]: from='client.15582 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:03:40.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:39 smithi161 ceph-mon[26055]: pgmap v169: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 2 op/s 2024-06-01T02:03:40.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:39 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/2084232182' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:40.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:39 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:03:40.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:39 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:03:40.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:39 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-01T02:03:40.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:39 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:40.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:40 smithi161 ceph-mon[26055]: Upgrade: Updating mgr.y 2024-06-01T02:03:40.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:40 smithi161 ceph-mon[26055]: Deploying daemon mgr.y on smithi116 2024-06-01T02:03:40.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:40 smithi161 ceph-mon[26055]: from='client.25492 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:03:40.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:40 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/4258698519' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:03:41.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:40 smithi116 ceph-mon[25082]: Upgrade: Updating mgr.y 2024-06-01T02:03:41.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:40 smithi116 ceph-mon[25082]: Deploying daemon mgr.y on smithi116 2024-06-01T02:03:41.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:40 smithi116 ceph-mon[25082]: from='client.25492 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:03:41.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:40 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/4258698519' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:03:41.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:40 smithi116 ceph-mon[29958]: Upgrade: Updating mgr.y 2024-06-01T02:03:41.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:40 smithi116 ceph-mon[29958]: Deploying daemon mgr.y on smithi116 2024-06-01T02:03:41.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:40 smithi116 ceph-mon[29958]: from='client.25492 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:03:41.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:40 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/4258698519' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:03:41.932 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:41 smithi116 ceph-mon[25082]: pgmap v170: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:03:41.933 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:41 smithi116 ceph-mon[29958]: pgmap v170: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:03:42.012 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:41 smithi161 ceph-mon[26055]: pgmap v170: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:03:43.695 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:43 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:03:43.505Z caller=manager.go:609 level=warn component="rule manager" group=nodes msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on(instance) group_left(nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi116\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:03:43.782 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:43 smithi116 systemd[1]: Stopping Ceph mgr.y for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:03:44.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:43 smithi116 ceph-mon[25082]: pgmap v171: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-06-01T02:03:44.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:43 smithi116 ceph-mon[29958]: pgmap v171: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-06-01T02:03:44.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:43 smithi161 ceph-mon[26055]: pgmap v171: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-06-01T02:03:44.404 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:44 smithi116 podman[77071]: 2024-06-01 02:03:44.120686877 +0000 UTC m=+0.131819951 container died a6cc56fad4e6a39612392508910924fed8925406f4dddb517e34125c6c9b1e9e (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-06-01T02:03:44.793 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:44 smithi116 podman[77071]: 2024-06-01 02:03:44.57327127 +0000 UTC m=+0.584404336 container cleanup a6cc56fad4e6a39612392508910924fed8925406f4dddb517e34125c6c9b1e9e (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-06-01T02:03:44.793 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:44 smithi116 bash[77071]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y 2024-06-01T02:03:45.644 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:45 smithi116 ceph-mon[29958]: pgmap v172: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:03:45.644 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:45 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:45.644 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:45 smithi116 ceph-mon[25082]: pgmap v172: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:03:45.645 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:45 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:45.645 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:45 smithi116 podman[77102]: 2024-06-01 02:03:45.453504898 +0000 UTC m=+1.332205720 container remove a6cc56fad4e6a39612392508910924fed8925406f4dddb517e34125c6c9b1e9e (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-06-01T02:03:45.645 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:45 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-06-01T02:03:45.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:45 smithi161 ceph-mon[26055]: pgmap v172: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:03:45.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:45 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:45.946 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:45 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-06-01T02:03:45.946 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:45 smithi116 systemd[1]: Stopped Ceph mgr.y for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:03:45.947 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:45 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mgr.y.service: Consumed 8.908s CPU time. 2024-06-01T02:03:45.947 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:45 smithi116 systemd[1]: Starting Ceph mgr.y for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:03:46.287 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:46 smithi116 podman[77395]: 2024-06-01 02:03:46.00963138 +0000 UTC m=+0.019781909 image pull e1d6a67b021eb077ee22bf650f1a9fb1980a2cf5c36bdb9cba9eac6de8f702d9 quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a 2024-06-01T02:03:46.287 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:46 smithi116 podman[77395]: 2024-06-01 02:03:46.146373059 +0000 UTC m=+0.156523615 container create 95437b8af57cd613239ce4aff5c7ea9ec4559cb44029df8ddd3c169ba72bf1f9 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y, version=8, com.redhat.component=centos-stream-container, io.k8s.display-name=CentOS Stream 8, build-date=2022-05-03T08:36:31.336870, io.buildah.version=1.19.8, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, GIT_CLEAN=True, distribution-scope=public, maintainer=Guillaume Abrioux , io.openshift.expose-services=, GIT_REPO=https://github.com/ceph/ceph-container.git, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., ceph=True, release=754, io.openshift.tags=base centos centos-stream, GIT_BRANCH=HEAD, vcs-type=git, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, vendor=Red Hat, Inc., description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, name=centos-stream, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, architecture=x86_64, RELEASE=HEAD, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, CEPH_POINT_RELEASE=-17.2.0) 2024-06-01T02:03:46.538 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:46 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:46.538 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:46 smithi116 ceph-mon[29958]: pgmap v173: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:03:46.538 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:46 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:46.538 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:46 smithi116 ceph-mon[29958]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:03:46.539 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:46 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:46.539 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:46 smithi116 ceph-mon[25082]: pgmap v173: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:03:46.539 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:46 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:46.539 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:46 smithi116 ceph-mon[25082]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:03:46.539 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:46 smithi116 podman[77395]: 2024-06-01 02:03:46.441084477 +0000 UTC m=+0.451234975 container init 95437b8af57cd613239ce4aff5c7ea9ec4559cb44029df8ddd3c169ba72bf1f9 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, com.redhat.component=centos-stream-container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, build-date=2022-05-03T08:36:31.336870, CEPH_POINT_RELEASE=-17.2.0, RELEASE=HEAD, vcs-type=git, architecture=x86_64, vendor=Red Hat, Inc., io.openshift.tags=base centos centos-stream, name=centos-stream, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, version=8, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, maintainer=Guillaume Abrioux , description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, distribution-scope=public, release=754, io.openshift.expose-services=, io.k8s.display-name=CentOS Stream 8, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, GIT_REPO=https://github.com/ceph/ceph-container.git, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, io.buildah.version=1.19.8) 2024-06-01T02:03:46.539 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:46 smithi116 podman[77395]: 2024-06-01 02:03:46.444002997 +0000 UTC m=+0.454153503 container start 95437b8af57cd613239ce4aff5c7ea9ec4559cb44029df8ddd3c169ba72bf1f9 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , com.redhat.component=centos-stream-container, GIT_CLEAN=True, distribution-scope=public, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, release=754, io.openshift.tags=base centos centos-stream, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, vcs-type=git, version=8, io.buildah.version=1.19.8, io.openshift.expose-services=, vendor=Red Hat, Inc., io.k8s.display-name=CentOS Stream 8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, GIT_REPO=https://github.com/ceph/ceph-container.git, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, name=centos-stream, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, RELEASE=HEAD, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., build-date=2022-05-03T08:36:31.336870, CEPH_POINT_RELEASE=-17.2.0, architecture=x86_64, GIT_BRANCH=HEAD, ceph=True) 2024-06-01T02:03:46.539 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:46 smithi116 ceph-mgr[77456]: -- 172.21.15.116:0/2019538182 <== mon.1 v2:172.21.15.116:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562637c02340 con 0x562637bdac00 2024-06-01T02:03:46.539 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:46 smithi116 bash[77395]: 95437b8af57cd613239ce4aff5c7ea9ec4559cb44029df8ddd3c169ba72bf1f9 2024-06-01T02:03:46.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:46 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:46.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:46 smithi161 ceph-mon[26055]: pgmap v173: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:03:46.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:46 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:46.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:46 smithi161 ceph-mon[26055]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:03:46.793 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:46 smithi116 systemd[1]: Started Ceph mgr.y for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:03:46.793 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:46 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[77438]: 2024-06-01T02:03:46.597+0000 7effea81a000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-01T02:03:46.794 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:46 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[77438]: 2024-06-01T02:03:46.739+0000 7effea81a000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-01T02:03:47.180 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:46 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[77438]: 2024-06-01T02:03:46.844+0000 7effea81a000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-01T02:03:47.180 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[77438]: 2024-06-01T02:03:47.183+0000 7effea81a000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-01T02:03:47.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:46 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:03:46] "GET /metrics HTTP/1.1" 200 38249 "" "Prometheus/2.33.4" 2024-06-01T02:03:47.436 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[77438]: 2024-06-01T02:03:47.268+0000 7effea81a000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-01T02:03:47.437 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-06-01T02:03:47.437 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: Cluster is now healthy 2024-06-01T02:03:47.437 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:47.437 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:47.437 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:47.437 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:47.437 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:47.437 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:47.437 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:47.438 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:47.438 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:47.438 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:47.438 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:47.438 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:47.438 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:47.438 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:47.438 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:47.438 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:47.438 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:47.438 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:47.438 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:47.438 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:47.439 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:47.439 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:47.439 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:47.439 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:47.439 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:47.439 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:47.439 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:47.439 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:47.439 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:47.439 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-06-01T02:03:47.439 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: Cluster is now healthy 2024-06-01T02:03:47.440 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:47.440 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:47.440 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:47.440 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:47.440 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:47.440 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:47.440 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:47.440 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:47.440 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:47.440 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:47.440 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:47.440 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:47.440 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:47.441 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:47.441 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:47.441 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:47.441 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:47.441 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:47.441 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:47.441 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:47.441 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:47.441 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:47.441 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:47.441 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:47.441 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:47.441 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:47.442 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:47.442 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:47.442 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:47.442 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:47.442 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:47.442 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:47.442 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:47.442 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:47.442 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:47.443 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:47.443 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:47.443 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:47.443 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:47.443 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:47.443 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:47.443 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:47.443 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:47.443 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:47.443 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:47.443 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:47.443 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:47.443 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:47.444 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:47 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:47.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-06-01T02:03:47.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: Cluster is now healthy 2024-06-01T02:03:47.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:47.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:47.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:47.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:47.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:47.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:47.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:47.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:47.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:47.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:47.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:47.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:47.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:47.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:47.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:47.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:47.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:47.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:47.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:47.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:47.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:47.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:47.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:47.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:47.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:47.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:47.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:47.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:47.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:47.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:47.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:47.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:47.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:47.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:47.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:47.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:47.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:47.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:47.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:47 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:48.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:48.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:48.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: pgmap v174: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-06-01T02:03:48.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:48.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:48.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:48.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:48.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:48.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:48.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:48.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:48.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:48.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:48.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:48.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:48.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:48.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:48.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:48.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:48.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:48.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:48.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:48.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:48.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:48.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:48.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:48.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:48.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:48.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:48.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:48.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:48.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:48.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:48.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:48.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:48.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:48.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:48.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:48.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:48.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:48.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:48.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:48.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:48.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:48.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:48.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:48.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:48.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:48.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:48.665 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[77438]: 2024-06-01T02:03:48.616+0000 7effea81a000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-01T02:03:48.665 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:48.666 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:48.666 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: pgmap v174: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-06-01T02:03:48.666 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:48.666 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:48.666 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:48.666 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:48.666 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:48.667 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:48.667 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:48.667 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:48.667 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:48.667 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:48.667 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:48.667 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:48.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:48.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:48.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:48.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:48.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:48.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:48.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:48.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:48.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:48.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:48.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:48.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:48.669 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:48.669 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:48.669 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:48.670 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:48.671 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:48.671 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: pgmap v174: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-06-01T02:03:48.671 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:48.671 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:48.671 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:48.671 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:48.672 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:48.672 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:48.672 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:48.672 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:48.672 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:48.672 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:48.673 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:48.673 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:48.673 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:48.673 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:48.673 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:48.673 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:48.673 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:48.673 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:48.673 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:48.673 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:48.673 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:48.673 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:48.673 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:48.674 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:48.674 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:48.674 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:48.674 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:48.674 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:48.674 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:48.674 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:48.674 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:48.674 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:48.675 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:48.675 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:48.675 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:48.675 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:48.675 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:48.675 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:48.675 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:48.675 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:48.675 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:48.675 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:48.675 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:48.675 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:48.675 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:48.675 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:48.676 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:48.676 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:48.676 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:48.676 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:48.676 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:48.676 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:48.676 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:48.676 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:48.676 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:48.676 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:48.677 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:48.677 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:48.677 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:48.677 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:48.677 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:48.677 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:48.677 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:48.677 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:48.677 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:48.944 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:48 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:03:48.660Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"172.21.15.161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"smithi161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:03:49.043 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[77438]: 2024-06-01T02:03:48.791+0000 7effea81a000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-01T02:03:49.043 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:48 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[77438]: 2024-06-01T02:03:48.878+0000 7effea81a000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-01T02:03:49.381 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[77438]: 2024-06-01T02:03:49.217+0000 7effea81a000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-01T02:03:49.381 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[77438]: 2024-06-01T02:03:49.313+0000 7effea81a000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-01T02:03:49.662 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[77438]: 2024-06-01T02:03:49.415+0000 7effea81a000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-01T02:03:49.663 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:49.663 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:49.663 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:49.663 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:49.663 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:49.663 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:49.663 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:49.663 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:49.663 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:49.663 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:49.663 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:49.663 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:49.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:49.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:49.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:49.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:49.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:49.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:49.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:49.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:49.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:49.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:49.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:49.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:49.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:49.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:49.665 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:49.665 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:49.665 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:49.665 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:49.665 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:49.665 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:49.665 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:49.665 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:49.665 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:49.665 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:49.665 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:49.665 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:49.665 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:49.666 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:49.666 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:49.666 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:49.666 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:49.666 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:49.666 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:49.666 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:49.666 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:49.666 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:49.666 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:49.667 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:49.667 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:49.667 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:49.667 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:49.667 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:49.667 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:49.667 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:49.667 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:49.667 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:49.667 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:49.667 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:49.667 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:49.667 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:49.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:49.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:49.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:49.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:49.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:49.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:49.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:49.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:49.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:49.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:49.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:49.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:49.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:49.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:49.669 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:49.669 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:49.669 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:49.669 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:49.669 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:49.669 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:49.669 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:49.669 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:49.669 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:49.669 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:49.669 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:49.669 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:49.669 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:49.670 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:49.670 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:49.670 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:49.670 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:49.670 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:49.670 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:49.670 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:49.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:49.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:49.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:49.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:49.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:49.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:49.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:49.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:49.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:49.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:49.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:49.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:49.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:49.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:49.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:49.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:49.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:49.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:49.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:49.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:49.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:49.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:49.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:49.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:49.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:49.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:49.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:49.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:49.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:49.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:49.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:49.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:49.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:49.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:49.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:49.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:49.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:49.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:49.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:49.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:49.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:49.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:49.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:49.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:49.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:49.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:49.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:49.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:49 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:50.017 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:49 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[77438]: 2024-06-01T02:03:49.666+0000 7effea81a000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-01T02:03:50.293 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[77438]: 2024-06-01T02:03:50.140+0000 7effea81a000 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-01T02:03:50.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:50.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:50.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:50.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: pgmap v175: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 965 B/s rd, 0 op/s 2024-06-01T02:03:50.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:50.665 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:50.665 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:50.665 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:50.665 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:50.665 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:50.665 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:50.665 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:50.665 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:50.665 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:50.665 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:50.665 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:50.665 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:50.665 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:50.665 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:50.666 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:50.666 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:50.666 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:50.666 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:50.666 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:50.666 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:50.666 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:50.666 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:50.666 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:50.666 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:50.666 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:50.667 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:50.667 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:50.667 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:50.667 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: pgmap v175: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 965 B/s rd, 0 op/s 2024-06-01T02:03:50.667 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:50.667 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:50.667 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:50.667 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:50.667 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:50.667 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:50.667 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:50.667 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:50.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:50.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:50.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:50.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:50.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:50.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:50.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:50.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:50.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:50.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:50.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:50.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:50.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:50.668 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:50.669 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:50.669 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:50.669 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:50.669 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:50.669 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:50.669 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:50.669 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:50.669 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:50.669 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:50.669 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:50.669 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:50.669 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:50.669 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:50.669 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:50.669 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:50.670 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:50.670 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:50.670 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:50.670 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:50.670 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:50.670 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:50.670 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:50.670 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:50.670 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:50.670 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:50.670 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:50.670 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:50.670 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:50.670 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:50.671 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:50.671 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:50.671 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:50.671 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:50.671 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:50.671 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:50.671 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:50.671 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:50.671 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:50.671 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:50.671 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:50.671 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:50.672 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:50.672 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:50.672 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:50.672 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:50.672 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:50.672 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:50.672 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:50.672 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:50.672 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:50.672 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:50.672 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:50.672 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:50.672 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:50.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:50.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:50.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:50.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: pgmap v175: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 965 B/s rd, 0 op/s 2024-06-01T02:03:50.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:50.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:50.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:50.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:50.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:50.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:50.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:50.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:50.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:50.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:50.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:50.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:50.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:50.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:50.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:50.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:50.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:50.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:50.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:50.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:50.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:50.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:50.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:50.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:50.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:50.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:50.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:50.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:50.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:50.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:50.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:50.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:50.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:50.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:50.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:50.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:50.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:50.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:50.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:50.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:50.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:50.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:50.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:50.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:50.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:50.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:50.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:50.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:50.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:50.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:50.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:50 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:50.946 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[77438]: 2024-06-01T02:03:50.667+0000 7effea81a000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-01T02:03:50.946 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:50 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[77438]: 2024-06-01T02:03:50.772+0000 7effea81a000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-01T02:03:51.262 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[77438]: 2024-06-01T02:03:51.085+0000 7effea81a000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-01T02:03:51.262 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[77438]: 2024-06-01T02:03:51.181+0000 7effea81a000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-01T02:03:51.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:51.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:51.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:51.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:51.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:51.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:51.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:51.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:51.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:51.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:51.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:51.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:51.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:51.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:51.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:51.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:51.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:51.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:51.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:51.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:51.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:51.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:51.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:51.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:51.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:51.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:51.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:51.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:51.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:51.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:51.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:51.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:51.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:51.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:51.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:51.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:51.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:51.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:51.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:51.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:51.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:51.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:51.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:51.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:51.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:51.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:51.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:51.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:51.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:51.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:51.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:51.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:51.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:51.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:51.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:51.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:51.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:51.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:51.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:51.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:51.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:51.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:51.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:51.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:51.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:51.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:51.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:51.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:51.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:51.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:51.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:51.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:51.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:51.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:51.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:51.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:51.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:51.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:51.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:51.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:51.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:51.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:51.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:51.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:51.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:51.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:51.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:51.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:51.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:51.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:51.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:51.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:51.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:51.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:51.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:51.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:51.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:51.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:51.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:51.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:51.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:51.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:51.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:51.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:51.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:51.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:51.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:51.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:51.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:51.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:51.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:51.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:51.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:51.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:51.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:51.554 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:51.554 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:51.554 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:51.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:51.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:51.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:51.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:51.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:51.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:51.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:51.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:51.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:51.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:51.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:51.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:51.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:51.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:51.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:51.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:51.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:51.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:51.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:51.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:51.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:51.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:51.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:51.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:51.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:51.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:51.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:51.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:51.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:51.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:51.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:51.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:51.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:51.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:51.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:51.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:51.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:51.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:51.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:51.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:51.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:51.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:51.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:51.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:51.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:51.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:51.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:51.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:51.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:51.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:51.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:51.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:51.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:51.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:51.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:51.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:51.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:51.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:51.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:51 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:51.862 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[77438]: 2024-06-01T02:03:51.613+0000 7effea81a000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-01T02:03:51.862 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[77438]: 2024-06-01T02:03:51.767+0000 7effea81a000 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-01T02:03:51.862 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[77438]: 2024-06-01T02:03:51.866+0000 7effea81a000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-01T02:03:52.185 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:51 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[77438]: 2024-06-01T02:03:51.998+0000 7effea81a000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-01T02:03:52.185 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[77438]: 2024-06-01T02:03:52.092+0000 7effea81a000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-01T02:03:52.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: pgmap v176: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 965 B/s rd, 0 op/s 2024-06-01T02:03:52.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:52.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:03:52.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:52.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:52.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:52.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:52.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:52.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:52.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:52.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:52.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:52.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:52.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:52.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:52.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:52.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:52.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:52.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:52.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:52.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:52.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:52.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:52.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: pgmap v176: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 965 B/s rd, 0 op/s 2024-06-01T02:03:52.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:52.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:03:52.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:52.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:52.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:52.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:52.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:52.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:52.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:52.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:52.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:52.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:52.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:52.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:52.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:52.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:52.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:52.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:52.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:52.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:52.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:52.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:52.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:52.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:52.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:52.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:52.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:52.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:52.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:52.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:52.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:52.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:52.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:52.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:52.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:52.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:52.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:52.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:52.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:52.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:52.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:52.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:52.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:52.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:52.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:52.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:52.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:52.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:52.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:52.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:52.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:52.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:52.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:52.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:52.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:52.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:52.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:52.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:52.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:52.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:52.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:52.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:52.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:52.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:52.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:52.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:52.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:52.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:52.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:52.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:52.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:52.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:52.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: pgmap v176: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 965 B/s rd, 0 op/s 2024-06-01T02:03:52.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:52.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:03:52.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:52.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:52.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:52.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:52.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:52.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:52.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:52.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:52.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:52.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:52.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:52.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:52.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:52.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:52.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:52.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:52.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:52.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:52.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:52.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:52.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:52.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:52.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:52.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:52.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:52.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:52.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:52.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:52.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:52.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:52.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:52.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:52.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:52.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:52.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:52.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:52.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:52.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:52.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:52.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:52.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:52.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:52.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:52.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:52.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:52.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:52 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:52.844 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:52 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[77438]: 2024-06-01T02:03:52.697+0000 7effea81a000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-01T02:03:53.189 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[77438]: 2024-06-01T02:03:53.086+0000 7effea81a000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-01T02:03:53.190 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[77438]: [01/Jun/2024:02:03:53] ENGINE Bus STARTING 2024-06-01T02:03:53.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:53.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:53.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:53.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:53.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:53.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:53.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:53.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:53.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:53.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:53.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:53.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:53.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:53.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:53.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:53.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:53.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:53.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:53.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:53.522 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:53.522 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:53.522 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:53.522 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:53.522 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:53.522 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:53.522 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:53.522 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:53.522 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:53.522 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:53.522 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[77438]: CherryPy Checker: 2024-06-01T02:03:53.523 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[77438]: The Application mounted at '' has an empty config. 2024-06-01T02:03:53.523 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[77438]: 2024-06-01T02:03:53.523 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[77438]: [01/Jun/2024:02:03:53] ENGINE Serving on http://:::9283 2024-06-01T02:03:53.523 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[77438]: [01/Jun/2024:02:03:53] ENGINE Bus STARTED 2024-06-01T02:03:53.523 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:53.523 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:53.523 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:53.523 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:53.523 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:53.523 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:53.524 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:53.524 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:53.524 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:53.524 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:53.524 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:53.524 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:53.524 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:53.524 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:53.524 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:53.524 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:53.524 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:53.524 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:53.524 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:53.524 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:53.524 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:53.525 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:53.525 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:53.525 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:53.525 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:53.525 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:53.525 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:53.525 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:53.525 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:53.525 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:53.525 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:53.525 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:53.526 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:53.526 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:53.526 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:53.526 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:53.526 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:53.526 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:53.526 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:53.526 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:53.526 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:53.526 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: Standby manager daemon y restarted 2024-06-01T02:03:53.526 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: Standby manager daemon y started 2024-06-01T02:03:53.526 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.116:0/2' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-01T02:03:53.526 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.116:0/2' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T02:03:53.526 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.116:0/2' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-01T02:03:53.527 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.116:0/2' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T02:03:53.527 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:53.527 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:53.527 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:53.527 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:53.527 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:53.527 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:53.527 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:53.527 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:53.527 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:53.527 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:53.528 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:53.528 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:53.528 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:53.528 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: Standby manager daemon y restarted 2024-06-01T02:03:53.528 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: Standby manager daemon y started 2024-06-01T02:03:53.528 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.116:0/2' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-01T02:03:53.528 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.116:0/2' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T02:03:53.528 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.116:0/2' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-01T02:03:53.528 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.116:0/2' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T02:03:53.528 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:53 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:53.695 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:03:53.505Z caller=manager.go:609 level=warn component="rule manager" group=nodes msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on(instance) group_left(nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi116\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:03:53.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:53.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:53.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:53.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:53.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:53.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:53.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:53.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:53.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:53.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:53.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:53.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:53.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:53.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:53.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:53.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:53.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:53.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:53.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:53.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:53.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:53.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:53.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:53.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:53.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:53.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:53.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:53.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:53.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:53.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:53.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:53.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:53.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:53.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:03:53.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:53.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:03:53.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:53.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:53.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:03:53.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:53.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:53.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: Standby manager daemon y restarted 2024-06-01T02:03:53.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: Standby manager daemon y started 2024-06-01T02:03:53.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.116:0/2' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-01T02:03:53.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.116:0/2' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T02:03:53.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.116:0/2' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-01T02:03:53.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.116:0/2' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T02:03:53.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:53 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:03:55.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:55 smithi116 ceph-mon[25082]: pgmap v177: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 965 B/s rd, 0 op/s 2024-06-01T02:03:55.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:55 smithi116 ceph-mon[25082]: mgrmap e37: x(active, since 47s), standbys: y 2024-06-01T02:03:55.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:55 smithi116 ceph-mon[29958]: pgmap v177: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 965 B/s rd, 0 op/s 2024-06-01T02:03:55.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:55 smithi116 ceph-mon[29958]: mgrmap e37: x(active, since 47s), standbys: y 2024-06-01T02:03:55.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:55 smithi161 ceph-mon[26055]: pgmap v177: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 965 B/s rd, 0 op/s 2024-06-01T02:03:55.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:55 smithi161 ceph-mon[26055]: mgrmap e37: x(active, since 47s), standbys: y 2024-06-01T02:03:56.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:56 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:56.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:56 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:56.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:56 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:56.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:56 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:56.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:56 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:56.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:56 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:56.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:56 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:56.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:56 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:56.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:56 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:56.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:56 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:56.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:56 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:56.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:56 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:56.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:56 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:56.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:56 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:56.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:56 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:56.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:56 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:56.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:56 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:56.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:56 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:57.092 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:03:56 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:03:56] "GET /metrics HTTP/1.1" 200 38242 "" "Prometheus/2.33.4" 2024-06-01T02:03:57.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:57 smithi116 ceph-mon[25082]: pgmap v178: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 966 B/s rd, 0 op/s 2024-06-01T02:03:57.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:57 smithi116 ceph-mon[25082]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:03:57.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:57 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:57.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:57 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:03:57.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:57 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:03:57.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:57 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-01T02:03:57.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:57 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:57.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:57 smithi116 ceph-mon[29958]: pgmap v178: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 966 B/s rd, 0 op/s 2024-06-01T02:03:57.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:57 smithi116 ceph-mon[29958]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:03:57.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:57 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:57.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:57 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:03:57.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:57 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:03:57.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:57 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-01T02:03:57.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:57 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:57.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:57 smithi161 ceph-mon[26055]: pgmap v178: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 966 B/s rd, 0 op/s 2024-06-01T02:03:57.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:57 smithi161 ceph-mon[26055]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:03:57.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:57 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:03:57.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:57 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:03:57.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:57 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:03:57.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:57 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-01T02:03:57.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:57 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:03:58.150 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:58 smithi116 ceph-mon[29958]: Upgrade: Updating mgr.y 2024-06-01T02:03:58.150 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:58 smithi116 ceph-mon[29958]: Deploying daemon mgr.y on smithi116 2024-06-01T02:03:58.151 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:58 smithi116 ceph-mon[25082]: Upgrade: Updating mgr.y 2024-06-01T02:03:58.151 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:58 smithi116 ceph-mon[25082]: Deploying daemon mgr.y on smithi116 2024-06-01T02:03:58.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:58 smithi161 ceph-mon[26055]: Upgrade: Updating mgr.y 2024-06-01T02:03:58.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:58 smithi161 ceph-mon[26055]: Deploying daemon mgr.y on smithi116 2024-06-01T02:03:58.718 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:58 smithi116 systemd[1]: Stopping Ceph mgr.y for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:03:58.906 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:03:58 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:03:58.660Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"172.21.15.161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"smithi161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:03:59.006 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:58 smithi116 podman[80583]: 2024-06-01 02:03:58.722463475 +0000 UTC m=+0.147153509 container died 95437b8af57cd613239ce4aff5c7ea9ec4559cb44029df8ddd3c169ba72bf1f9 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y, distribution-scope=public, version=8, CEPH_POINT_RELEASE=-17.2.0, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, com.redhat.component=centos-stream-container, name=centos-stream, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, ceph=True, RELEASE=HEAD, io.openshift.expose-services=, vcs-type=git, release=754, io.k8s.display-name=CentOS Stream 8, GIT_BRANCH=HEAD, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, io.buildah.version=1.19.8, GIT_CLEAN=True, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, architecture=x86_64, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, GIT_REPO=https://github.com/ceph/ceph-container.git, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, io.openshift.tags=base centos centos-stream, maintainer=Guillaume Abrioux , url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vendor=Red Hat, Inc., build-date=2022-05-03T08:36:31.336870) 2024-06-01T02:03:59.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:03:59 smithi161 ceph-mon[26055]: pgmap v179: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:03:59.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:03:59 smithi116 ceph-mon[25082]: pgmap v179: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:03:59.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:03:59 smithi116 ceph-mon[29958]: pgmap v179: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:03:59.294 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:59 smithi116 podman[80583]: 2024-06-01 02:03:59.100073757 +0000 UTC m=+0.524763784 container cleanup 95437b8af57cd613239ce4aff5c7ea9ec4559cb44029df8ddd3c169ba72bf1f9 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y, GIT_CLEAN=True, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., RELEASE=HEAD, release=754, io.openshift.expose-services=, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, build-date=2022-05-03T08:36:31.336870, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, vcs-type=git, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, vendor=Red Hat, Inc., name=centos-stream, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, GIT_BRANCH=HEAD, com.redhat.component=centos-stream-container, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, architecture=x86_64, distribution-scope=public, io.k8s.display-name=CentOS Stream 8, ceph=True, version=8, CEPH_POINT_RELEASE=-17.2.0, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, io.openshift.tags=base centos centos-stream, io.buildah.version=1.19.8) 2024-06-01T02:03:59.294 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:59 smithi116 bash[80583]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y 2024-06-01T02:03:59.692 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:59 smithi116 podman[80596]: 2024-06-01 02:03:59.351791918 +0000 UTC m=+0.626306049 container remove 95437b8af57cd613239ce4aff5c7ea9ec4559cb44029df8ddd3c169ba72bf1f9 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, distribution-scope=public, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, release=754, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, RELEASE=HEAD, io.buildah.version=1.19.8, build-date=2022-05-03T08:36:31.336870, io.openshift.expose-services=, version=8, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, com.redhat.component=centos-stream-container, io.k8s.display-name=CentOS Stream 8, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, CEPH_POINT_RELEASE=-17.2.0, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, architecture=x86_64, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vendor=Red Hat, Inc., ceph=True, vcs-type=git, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, name=centos-stream, io.openshift.tags=base centos centos-stream, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-06-01T02:03:59.692 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:59 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-06-01T02:03:59.692 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:59 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-06-01T02:03:59.692 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:59 smithi116 systemd[1]: Stopped Ceph mgr.y for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:03:59.692 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:59 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mgr.y.service: Consumed 7.916s CPU time. 2024-06-01T02:03:59.692 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:59 smithi116 systemd[1]: Starting Ceph mgr.y for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:04:00.043 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:59 smithi116 podman[80725]: 2024-06-01 02:03:59.81135778 +0000 UTC m=+0.016549898 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:04:00.043 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:03:59 smithi116 podman[80725]: 2024-06-01 02:03:59.944565793 +0000 UTC m=+0.149757893 container create bc9627c27a5790b781a019c6e738e7cd47dfca82f60f338b37f8fa5f1058fcfd (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True) 2024-06-01T02:04:00.336 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:00 smithi116 podman[80725]: 2024-06-01 02:04:00.179480673 +0000 UTC m=+0.384672773 container init bc9627c27a5790b781a019c6e738e7cd47dfca82f60f338b37f8fa5f1058fcfd (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True) 2024-06-01T02:04:00.336 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:00 smithi116 podman[80725]: 2024-06-01 02:04:00.182346474 +0000 UTC m=+0.387538575 container start bc9627c27a5790b781a019c6e738e7cd47dfca82f60f338b37f8fa5f1058fcfd (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_CLEAN=True, ceph=True) 2024-06-01T02:04:00.337 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:00 smithi116 ceph-mgr[80745]: -- 172.21.15.116:0/2852768115 <== mon.1 v2:172.21.15.116:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56346b7285a0 con 0x56346b6fd000 2024-06-01T02:04:00.337 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:00 smithi116 bash[80725]: bc9627c27a5790b781a019c6e738e7cd47dfca82f60f338b37f8fa5f1058fcfd 2024-06-01T02:04:00.337 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:00 smithi116 systemd[1]: Started Ceph mgr.y for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:04:00.638 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:00 smithi116 ceph-mon[25082]: pgmap v180: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:00.638 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:00 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:00.638 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:00 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:00.639 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:00 smithi116 ceph-mon[29958]: pgmap v180: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:00.639 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:00 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:00.639 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:00 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:00.639 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:00 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:00.339+0000 7ffb0054b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-01T02:04:00.640 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:00 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:00.533+0000 7ffb0054b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-01T02:04:00.640 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:00 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:00.641+0000 7ffb0054b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-01T02:04:00.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:00 smithi161 ceph-mon[26055]: pgmap v180: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:00.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:00 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:00.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:00 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:01.293 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:00 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:00.968+0000 7ffb0054b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-01T02:04:01.293 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:01.058+0000 7ffb0054b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-01T02:04:01.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:01.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:01.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:01.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:01.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:01.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:01.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:01.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-06-01T02:04:01.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-06-01T02:04:01.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]': finished 2024-06-01T02:04:01.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:01.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:01.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:01.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:01.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:01.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:01.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:01.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:01.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:01.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:01.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:01.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:01.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:01.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:01.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:01.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:01.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:01.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:01.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:01.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:01.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:01.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:01.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:01.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:01.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:01.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:01.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:01.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:01.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:01.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:01.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:01.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:01.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:01.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:01.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:01.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:01.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:01.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:01.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:01.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:01.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:01.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:01.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:01.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:01.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:01.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:01.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:01.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:01.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:01.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:01.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:01.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:01.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:01.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:01.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:01.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:01 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:01.700 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:01.700 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:01.700 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:01.700 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:01.700 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:01.700 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:01.700 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:01.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-06-01T02:04:01.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-06-01T02:04:01.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]': finished 2024-06-01T02:04:01.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:01.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:01.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:01.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:01.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:01.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:01.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:01.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:01.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:01.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:01.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:01.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:01.702 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:01.702 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:01.702 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:01.702 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:01.702 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:01.702 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:01.702 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:01.702 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:01.702 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:01.702 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:01.703 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:01.703 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:01.703 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:01.703 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:01.703 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:01.703 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-06-01T02:04:01.703 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-06-01T02:04:01.703 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]': finished 2024-06-01T02:04:01.703 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:01.703 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:01.703 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:01.703 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:01.703 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:01.703 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:01.703 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:01.704 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:01.704 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:01.704 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:01.704 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:01.704 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:01.704 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:01.704 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:01.704 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:01.704 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:01.704 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:01.704 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:01.704 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:01.704 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:01.705 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:01.705 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:01.705 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:01.705 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:01.705 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:01.705 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:01.705 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:01.705 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:01.705 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:01.705 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:01.706 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:01.706 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:01.706 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:01.706 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:01.706 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:01.706 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:01.706 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:01.706 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:01.706 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:01.706 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:01.706 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:01.706 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:01.706 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:01.706 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:01.707 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:01.707 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:01.707 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:01.707 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:01.707 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:01.707 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:01.707 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:01.707 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:01.707 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:01.707 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:01.707 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:01.708 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:01.708 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:01.708 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:01.708 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:01.708 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:01.708 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:01.708 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:01.708 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:01.708 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:01.708 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:01.708 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:01.708 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:01.709 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:01.709 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:01.709 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:01.710 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:01.710 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:01.710 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:01.710 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:01.710 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:01.710 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:01.710 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:01.710 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:01.710 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:01.710 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:01.711 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:01.711 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:01.711 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:01.711 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:01.712 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:01.712 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:01.712 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:01.712 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:01.712 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:01.712 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:01.712 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:01.712 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:01 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:02.605 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:02.605 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:02.605 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:02.605 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:02.605 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: pgmap v181: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:02.605 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:02.605 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:02.605 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:02.605 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:02.605 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:02.606 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:02.606 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:02.606 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:02.606 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:02.606 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:02.606 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:02.606 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:02.606 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:02.606 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:02.606 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:02.606 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:02.606 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:02.606 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:02.607 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:02.607 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:02.607 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:02.607 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:02.607 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:02.607 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:02.607 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:02.607 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:02.337+0000 7ffb0054b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-01T02:04:02.607 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:02.521+0000 7ffb0054b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-01T02:04:02.608 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:02.608 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:02.608 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:02.608 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:02.608 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: pgmap v181: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:02.608 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:02.608 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:02.608 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:02.608 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:02.608 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:02.608 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:02.609 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:02.609 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:02.609 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:02.609 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:02.609 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:02.609 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:02.609 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:02.609 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:02.609 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:02.609 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:02.609 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:02.609 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:02.609 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:02.609 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:02.610 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:02.610 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:02.610 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:02.610 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:02.610 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:02.610 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:02.610 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:02.610 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:02.611 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:02.611 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:02.611 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:02.611 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:02.611 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:02.611 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:02.611 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:02.611 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:02.611 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:02.611 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:02.611 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:02.611 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:02.611 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:02.612 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:02.612 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:02.612 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:02.612 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:02.612 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:02.612 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:02.612 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:02.612 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:02.612 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:02.612 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:02.612 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:02.612 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:02.612 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:02.612 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:02.613 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:02.613 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:02.613 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:02.613 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:02.613 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:02.613 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:02.613 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:02.613 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:02.613 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:02.613 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:02.614 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:02.614 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:02.614 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:02.614 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:02.614 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:02.614 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:02.614 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:02.614 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:02.614 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:02.614 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:02.614 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:02.614 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:02.614 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:02.615 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:02.615 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:02.617 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:02.617 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:02.617 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:02.617 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:02.617 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:02.617 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:02.617 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:02.618 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:02.618 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:02.618 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:02.618 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:02.618 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:02.618 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:02.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:02.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:02.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:02.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:02.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: pgmap v181: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:02.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:02.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:02.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:02.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:02.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:02.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:02.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:02.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:02.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:02.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:02.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:02.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:02.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:02.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:02.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:02.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:02.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:02.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:02.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:02.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:02.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:02.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:02.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:02.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:02.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:02.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:02.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:02.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:02.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:02.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:02.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:02.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:02.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:02.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:02.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:02.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:02.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:02.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:02.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:02.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:02.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:02.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:02.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:02.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:02.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:02.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:02.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:02.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:02.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:02.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:02.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:02.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:02.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:02.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:02.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:02.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:02.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:02.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:02.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:02 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:02.878 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:02 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:02.701+0000 7ffb0054b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-01T02:04:03.138 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:03.042+0000 7ffb0054b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-01T02:04:03.139 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:03.142+0000 7ffb0054b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-01T02:04:03.544 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:03.356+0000 7ffb0054b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-01T02:04:03.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:03.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:03.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:03.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:03.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:03.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:03.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:03.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:03.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:03.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:03.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:03.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:03.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:03.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:03.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:03.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:03.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:03.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:03.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:03.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:03.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:03.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:03.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:03.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:03.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:03.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:03.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:03.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:03.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:03.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:03.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:03.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:03.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:03.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:03.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:03.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:03.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:03.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:03.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:03.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:03.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:03.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:03.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:03.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:03.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:03.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:03.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:03.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:03.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:03.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:03.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:03.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:03.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:03.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:03.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:03.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:03.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:03.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:03.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:03.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:03.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:03.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:03.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:03.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:03.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:03.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:03.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:03.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:03.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:03.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:03.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:03.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:03.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:03.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:03.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:03.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:03.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:03.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:03.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:03.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:03.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:03.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:03.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:03.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:03.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:03.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:03.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:03.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:03.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:03.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:03.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:03.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:03.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:03.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:03.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:03.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:03.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:03.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:03.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:03.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:03.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:03.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:03.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:03.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:03.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:03.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:03.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:03.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:03.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:03.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:03.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:03.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:03.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:03.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:03.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:03.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:03.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:03.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:03.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:03.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:03.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:03.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:03.694 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:04:03.505Z caller=manager.go:609 level=warn component="rule manager" group=nodes msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on(instance) group_left(nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi116\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:04:03.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:03.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:03.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:03.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:03.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:03.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:03.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:03.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:03.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:03.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:03.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:03.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:03.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:03.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:03.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:03.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:03.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:03.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:03.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:03.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:03.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:03.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:03.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:03.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:03.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:03.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:03.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:03.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:03.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:03.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:03.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:03.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:03.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:03.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:03.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:03.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:03.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:03.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:03.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:03.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:03.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:03.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:03.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:03.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:03.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:03.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:03.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:03.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:03.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:03.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:03.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:03.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:03.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:03.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:03.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:03.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:03.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:03.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:03.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:03.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:03.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:03 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:03.841 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:03 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:03.584+0000 7ffb0054b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-01T02:04:04.499 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:04.242+0000 7ffb0054b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-01T02:04:04.500 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:04.500 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:04.500 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:04.500 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:04.500 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:04.500 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:04.500 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: pgmap v182: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:04:04.500 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:04.500 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:04.500 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:04.500 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:04.500 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:04.500 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:04.501 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:04.501 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:04.501 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:04.501 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:04.501 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:04.501 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:04.501 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:04.501 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:04.501 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:04.502 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:04.502 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:04.502 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:04.502 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:04.502 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:04.502 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:04.503 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:04.503 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:04.503 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:04.503 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:04.503 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:04.504 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:04.504 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:04.504 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:04.504 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:04.504 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: pgmap v182: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:04:04.504 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:04.504 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:04.504 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:04.504 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:04.504 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:04.504 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:04.504 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:04.504 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:04.505 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:04.505 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:04.505 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:04.505 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:04.505 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:04.506 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:04.506 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:04.506 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:04.506 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:04.506 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:04.506 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:04.506 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:04.506 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:04.506 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:04.506 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:04.506 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:04.507 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:04.507 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:04.507 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:04.507 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:04.507 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:04.507 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:04.507 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:04.507 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:04.507 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:04.507 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:04.507 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:04.507 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:04.508 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:04.508 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:04.508 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:04.508 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:04.508 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:04.508 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:04.508 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:04.508 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:04.508 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:04.508 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:04.508 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:04.508 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:04.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:04.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:04.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:04.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:04.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:04.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:04.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:04.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:04.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:04.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:04.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:04.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:04.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:04.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:04.510 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:04.510 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:04.510 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:04.510 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:04.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:04.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:04.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:04.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:04.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:04.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:04.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: pgmap v182: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:04:04.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:04.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:04.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:04.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:04.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:04.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:04.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:04.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:04.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:04.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:04.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:04.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:04.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:04.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:04.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:04.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:04.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:04.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:04.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:04.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:04.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:04.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:04.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:04.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:04.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:04.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:04.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:04.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:04.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:04.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:04 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:04.793 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:04.695+0000 7ffb0054b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-01T02:04:05.092 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:04 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:04.797+0000 7ffb0054b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-01T02:04:05.092 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:05.019+0000 7ffb0054b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-01T02:04:05.673 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:05.577+0000 7ffb0054b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-01T02:04:05.673 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:05.676+0000 7ffb0054b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-01T02:04:05.673 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:05.673 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:05.674 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:05.674 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:05.674 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:05.674 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:05.674 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:05.674 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:05.674 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:05.674 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:05.674 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:05.674 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:05.674 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:05.674 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:05.674 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:05.674 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:05.675 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:05.675 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:05.675 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:05.675 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:05.675 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:05.675 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:05.675 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:05.675 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:05.675 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:05.675 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:05.675 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:05.675 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:05.675 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:05.675 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:05.676 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:05.676 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:05.676 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:05.676 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:05.676 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:05.676 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:05.676 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:05.676 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:05.676 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:05.676 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:05.676 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:05.677 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:05.677 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:05.677 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:05.677 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:05.677 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:05.677 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:05.677 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:05.677 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:05.677 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:05.677 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:05.677 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:05.677 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:05.677 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:05.677 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:05.678 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:05.678 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:05.678 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:05.678 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:05.678 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:05.678 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:05.678 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:05.678 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:05.678 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:05.678 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:05.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:05.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:05.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:05.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:05.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:05.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:05.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:05.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:05.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:05.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:05.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:05.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:05.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:05.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:05.680 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:05.680 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:05.680 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:05.680 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:05.680 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:05.680 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:05.680 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:05.680 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:05.680 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:05.680 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:05.680 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:05.681 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:05.681 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:05.681 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:05.681 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:05.681 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:05.681 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:05.681 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:05.681 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:05.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:05.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:05.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:05.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:05.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:05.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:05.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:05.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:05.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:05.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:05.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:05.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:05.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:05.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:05.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:05.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:05.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:05.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:05.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:05.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:05.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:05.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:05.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:05.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:05.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:05.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:05.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:05.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:05.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:05.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:05.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:05.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:05.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:05.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:05.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:05.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:05.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:05.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:05.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:05.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:05.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:05.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:05.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:05.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:05.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:05.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:05.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:05.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:05.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:05 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:06.235 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:05 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:05.943+0000 7ffb0054b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-01T02:04:06.235 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:06.087+0000 7ffb0054b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-01T02:04:06.235 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:06.193+0000 7ffb0054b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-01T02:04:06.543 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:06.324+0000 7ffb0054b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-01T02:04:06.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:06.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:06.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:06.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:06.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: pgmap v183: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:06.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:06.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:06.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:06.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:06.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:06.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:06.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:06.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:06.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:06.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:06.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:06.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:06.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:06.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:06.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:06.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:06.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:06.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:06.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:06.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:06.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:06.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:06.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:06.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:06.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:06.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:06.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:06.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:06.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:06.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: pgmap v183: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:06.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:06.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:06.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:06.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:06.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:06.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:06.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:06.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:06.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:06.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:06.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:06.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:06.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:06.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:06.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:06.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:06.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:06.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:06.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:06.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:06.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:06.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:06.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:06.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:06.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:06.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:06.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:06.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:06.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:06.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:06.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:06.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:06.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:06.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:06.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:06.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:06.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:06.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:06.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:06.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:06.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:06.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:06.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:06.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:06.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:06.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:06.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:06.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:06.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:06.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:04:06.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:06.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:06.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:06.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:06.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:06.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:06.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:06.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:06.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:06.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:06.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:06.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:06.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:06.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:06.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:06.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:06.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:06.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:06.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:06.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:06.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:06.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:06.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:06.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:06.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:06.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:06.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:06.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:06.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:06.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:06.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:06.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:06.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:06.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:06.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:06.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:06.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:06.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:06.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:06.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:04:06.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:06.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:06.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:06.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:06.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:06.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:06.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:06.555 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:06.555 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:06.555 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:06.555 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:06.555 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:06.555 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:06.555 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:06.555 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:06.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:06.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:06.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:06.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:06.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: pgmap v183: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:06.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:06.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:06.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:06.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:06.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:06.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:06.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:06.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:06.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:06.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:06.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:06.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:06.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:06.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:06.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:06.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:06.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:06.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:06.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:06.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:06.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:06.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:06.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:06.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:06.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:06.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:06.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:06.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:06.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:06.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:06.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:06.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:06.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:06.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:06.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:06.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:06.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:06.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:06.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:06.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:06.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:06.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:06.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:06.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:06.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:06.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:06.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:06.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:06.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:06.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:04:06.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:06.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:06.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:06.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:06.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:06.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:06.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:06.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:06.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:06.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:06.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:06.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:06.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:06.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:06.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:06.922 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:06 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:06.645+0000 7ffb0054b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-01T02:04:07.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:04:06] "GET /metrics HTTP/1.1" 200 38245 "" "Prometheus/2.33.4" 2024-06-01T02:04:07.543 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:07.320+0000 7ffb0054b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-01T02:04:07.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:07.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:07.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:07.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:07.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:07.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:07.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:07.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:07.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:07.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:07.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:07.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:07.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:04:07.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:07.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:07.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:07.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:07.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:07.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:07.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:07.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:07.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:07.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:07.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:07.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:07.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:07.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:07.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:07.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:07.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:07.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:07.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:07.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:07.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:07.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:07.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:07.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:07.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:07.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:07.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:07.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:07.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:07.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:07.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:04:07.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:07.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:07.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:07.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:07.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:07.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:07.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:07.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:07.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:07.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:07.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:07.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:07.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:07.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:07.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:07.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:07.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:07.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:07.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:07.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:07.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:07.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:07.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:07.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:07.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:07.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:07.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:07.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:07.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:07.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:07.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:07.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:07.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:07.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:07.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:07.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:07.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:07.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:07.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:07.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:07.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:07.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:07.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:07.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:07.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:07.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:07.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:07.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:07.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:07.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:07.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:07.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:07.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:07.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:07.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:07.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:07.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:07.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:07.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:07.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:07.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:07.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:07.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:07.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:07.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:07.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:07.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:07.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:07.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:07.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:07.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:07.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:07.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:07.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:07.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:07.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:07.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:07.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:07.554 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:07.554 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:07.554 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:07.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:07.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:07.555 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:07.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:07.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:07.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:07.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:07.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:07.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:07.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:07.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:07.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:07.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:07.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:07.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:07.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:04:07.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:07.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:07.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:07.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:07.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:07.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:07.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:07.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:07.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:07.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:07.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:07.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:07.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:07.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:07.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:07.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:07.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:07.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:07.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:07.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:07.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:07.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:07.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:07.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:07.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:07.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:07.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:07.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:07.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:07.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:07.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:07.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:07.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:07.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:07.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:07.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:07.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:07.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:07.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:07.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:07.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:07.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:07.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:07.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:07.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:07.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:07.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:07.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:07.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:07.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:07.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:07 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:08.035 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:07.579+0000 7ffb0054b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-01T02:04:08.035 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: [01/Jun/2024:02:04:07] ENGINE Bus STARTING 2024-06-01T02:04:08.035 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: CherryPy Checker: 2024-06-01T02:04:08.035 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: The Application mounted at '' has an empty config. 2024-06-01T02:04:08.035 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:08.036 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: [01/Jun/2024:02:04:07] ENGINE Serving on http://:::9283 2024-06-01T02:04:08.036 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:07 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: [01/Jun/2024:02:04:07] ENGINE Bus STARTED 2024-06-01T02:04:08.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:08.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:08.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:08.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:08.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:08.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:08.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: pgmap v184: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:04:08.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:08.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:08.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:08.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:08.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:08.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:08.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:08.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:08.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:08.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:08.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:08.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:08.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:08.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:08.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: Standby manager daemon y restarted 2024-06-01T02:04:08.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: Standby manager daemon y started 2024-06-01T02:04:08.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.116:0/892229294' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-01T02:04:08.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.116:0/892229294' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T02:04:08.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.116:0/892229294' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-01T02:04:08.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.116:0/892229294' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T02:04:08.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:08.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:08.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:08.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:08.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:08.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:08.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:08.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:08.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:08.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:08.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:08.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:08.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:08.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:08.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:08.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:08.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:08.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:08.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:08.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:08.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:08.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:08.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:08.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:08.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:08.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:08.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:08.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:08.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:08.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:08.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:08.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:08.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:08.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:08.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:08.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:08.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:08.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:08.759 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:08.759 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:08.760 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:08.760 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:08.760 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:08.760 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:08.760 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: pgmap v184: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:04:08.760 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:08.760 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:08.760 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:08.760 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:08.760 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:08.760 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:08.760 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:08.760 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:08.760 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:08.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:08.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:08.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:08.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:08.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:08.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: Standby manager daemon y restarted 2024-06-01T02:04:08.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: Standby manager daemon y started 2024-06-01T02:04:08.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.116:0/892229294' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-01T02:04:08.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.116:0/892229294' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T02:04:08.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.116:0/892229294' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-01T02:04:08.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.? 172.21.15.116:0/892229294' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T02:04:08.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:08.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:08.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:08.762 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:08.762 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:08.762 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:08.762 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:08.762 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:08.762 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:08.762 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:08.762 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: pgmap v184: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:04:08.762 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:08.763 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:08.763 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:08.763 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:08.763 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:08.763 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:08.763 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:08.763 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:08.763 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:08.763 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:08.763 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:08.763 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:08.763 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:08.763 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:08.763 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: Standby manager daemon y restarted 2024-06-01T02:04:08.766 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: Standby manager daemon y started 2024-06-01T02:04:08.766 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.116:0/892229294' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-01T02:04:08.766 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.116:0/892229294' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T02:04:08.766 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.116:0/892229294' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-01T02:04:08.766 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.116:0/892229294' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T02:04:08.766 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:08.766 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:08.766 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:08.766 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:08.767 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:08.767 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:08.767 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:08.767 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:08.767 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:08.767 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:08.767 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:08.767 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:08.767 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:08.768 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:08.768 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:08.768 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:08.768 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:08.768 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:08.768 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:08.768 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:08.768 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:08.768 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:08.768 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:08.768 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:08.768 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:08.768 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:08.768 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:08.769 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:08.769 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:08.769 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:08.769 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:08.769 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:08.769 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:08.769 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:08.769 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:08.769 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:08.769 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:08.769 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:08.769 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:08.770 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:08.770 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:08.770 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:08.770 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:08.770 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:08.770 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:08.770 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:08.770 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:08.770 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:08.770 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:08.770 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:08.770 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:08.771 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:08.771 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:08.771 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:08.771 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:08.771 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:08.771 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:08.771 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:08.771 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:08.771 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:08.771 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:08.771 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:08.771 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:08.771 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:08.771 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:08.771 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:08.771 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:08.772 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:08.772 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:08.772 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:08.772 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:08.772 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:08 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:08.944 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:04:08 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:04:08.660Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"172.21.15.161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"smithi161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:04:09.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:09.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:09.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:09.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:09.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:09.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:09.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:09.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:09.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:09.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:09.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:09.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:09.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:09.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:09.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:09.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:09.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:09.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:09.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: mgrmap e38: x(active, since 62s), standbys: y 2024-06-01T02:04:09.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:09.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:09.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:09.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:09.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:09.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:09.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:09.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:09.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:09.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:09.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:09.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:09.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:09.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:09.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:09.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:09.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:09.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:09.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:09.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:09.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:09.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:09.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:09.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:09.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:09.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:09.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:09.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:09.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:09.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:09.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:09.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:09.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:09.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:09.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:09.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:09.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:09.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:09.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:09.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:09 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:09.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:09.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:09.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:09.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:09.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:09.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:09.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:09.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:09.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:09.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:09.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:09.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:09.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:09.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:09.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:09.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:09.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:09.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:09.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: mgrmap e38: x(active, since 62s), standbys: y 2024-06-01T02:04:09.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:09.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:09.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:09.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:09.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:09.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:09.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:09.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:09.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:09.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:09.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:09.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:09.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:09.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:09.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:09.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:09.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:09.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:09.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:09.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:09.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:09.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:09.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:09.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:09.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:09.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-2946-g3d89accf 2024-06-01T02:04:09.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:09.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:09.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:09.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:09.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: mgrmap e38: x(active, since 62s), standbys: y 2024-06-01T02:04:09.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:09.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:09.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:09.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:09.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:09.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:09.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:09.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:09.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:09.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:09.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:09.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:09.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:09.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:09.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:09.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:09.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:09.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:09.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:09.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:09.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:09.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:09.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:09.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:09.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:09.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:09.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:09.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:09.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:09.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:09.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:09.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:09.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:09.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:09.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:09.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:09.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:09.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:09.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:09.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:09.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:09.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:09.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:09.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:09.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:09.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:09.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:09.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:09.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:09.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:09.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:09.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:09.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:09.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:09.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:09.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:09.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:09.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:09.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:09.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:09.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:09.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:09.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:09.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:09.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:09.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:09.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:09.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:09 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:10.375 INFO:teuthology.orchestra.run.smithi116.stdout:true 2024-06-01T02:04:10.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:10.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:10.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: pgmap v185: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:10.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:10.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:10.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:10.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:10.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:10.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:10.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:10.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:10.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:10.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:10.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:10.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:10.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:10.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:10.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:10.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:10.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:10.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:10.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:10.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:10.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:10.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:10.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:10.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:10.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:10.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:10.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:10.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:10.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:10.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: pgmap v185: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:10.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:10.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:10.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:10.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:10.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:10.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:10.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:10.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:10.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:10.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:10.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:10.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:10.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:10.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:10.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:10.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:10.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:10.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:10.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:10.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:10.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:10.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:10.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:10.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:10.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:10.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:10.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:10.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:10.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:10.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:10.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:10.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:10.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:10.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:10.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:10.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:10.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:10.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:10.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:10.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:10.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:10.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:10.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:10.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:10.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:10.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:10.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:10.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:10.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:10.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:10.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:10.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:10.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:10.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:10.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:10.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:10.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:10.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:10.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:10.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:10.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:10.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:10.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:10.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:10.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:10.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:10.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:10.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:10.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:10.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:10.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:10.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:10.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:10.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:10.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:10 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:10.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:10.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:10.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: pgmap v185: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:10.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:10.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:10.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:10.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:10.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:10.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:10.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:10.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:10.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:10.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:10.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:10.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:10.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:10.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:10.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:10.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:10.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:10.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:10.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:10.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:10.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:10.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:10.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:10.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:10.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:10.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:10.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:10.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:10.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:10.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:10.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:10.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:10.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:10.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:10.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:10.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:10.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:10.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:10.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:10.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:10.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:10.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:10.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:10.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:10.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:10.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:10.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:10.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:10.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:10.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:10.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:10 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:11.052 INFO:teuthology.orchestra.run.smithi116.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-01T02:04:11.052 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi116 smithi116 running 15s ago 12m - - 2024-06-01T02:04:11.052 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi161 smithi161 running 15s ago 12m - - 2024-06-01T02:04:11.052 INFO:teuthology.orchestra.run.smithi116.stdout:alertmanager.a smithi116 *:9093,9094 running (2m) 15s ago 7m 15.5M - 0.23.0 ba2b418f427c e0c297162ded 2024-06-01T02:04:11.052 INFO:teuthology.orchestra.run.smithi116.stdout:grafana.a smithi161 *:3000 running (108s) 15s ago 7m 44.5M - 8.3.5 dad864ee21e9 00b0a78ee88e 2024-06-01T02:04:11.052 INFO:teuthology.orchestra.run.smithi116.stdout:iscsi.foo.smithi116.gjuxzb smithi116 running (55s) 15s ago 5m 46.3M - 3.5 e1d6a67b021e 0e191fe3e3bb 2024-06-01T02:04:11.052 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.x smithi161 *:8443,9283,8765 running (72s) 15s ago 11m 509M - 19.0.0-2946-g3d89accf b045ac545f82 2d7fbb85e5f7 2024-06-01T02:04:11.052 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.y smithi116 *:8443,9283,8765 starting - - - - 2024-06-01T02:04:11.052 INFO:teuthology.orchestra.run.smithi116.stdout:mon.a smithi116 running (13m) 15s ago 13m 66.1M 2048M 17.2.0 e1d6a67b021e c821092a538a 2024-06-01T02:04:11.052 INFO:teuthology.orchestra.run.smithi116.stdout:mon.b smithi161 running (12m) 15s ago 12m 50.2M 2048M 17.2.0 e1d6a67b021e 9011986aa6e3 2024-06-01T02:04:11.052 INFO:teuthology.orchestra.run.smithi116.stdout:mon.c smithi116 running (12m) 15s ago 12m 51.2M 2048M 17.2.0 e1d6a67b021e 4a6fe0b16d63 2024-06-01T02:04:11.053 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.a smithi116 *:9100 running (2m) 15s ago 7m 17.1M - 1.3.1 1dbe0e931976 e347748d17bb 2024-06-01T02:04:11.053 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.b smithi161 *:9100 running (112s) 15s ago 7m 19.3M - 1.3.1 1dbe0e931976 6e66182f5a1a 2024-06-01T02:04:11.053 INFO:teuthology.orchestra.run.smithi116.stdout:osd.0 smithi116 running (11m) 15s ago 11m 54.8M 4096M 17.2.0 e1d6a67b021e 0cc929b0ecad 2024-06-01T02:04:11.053 INFO:teuthology.orchestra.run.smithi116.stdout:osd.1 smithi116 running (11m) 15s ago 11m 55.7M 4096M 17.2.0 e1d6a67b021e 6ff7fc5abe58 2024-06-01T02:04:11.053 INFO:teuthology.orchestra.run.smithi116.stdout:osd.2 smithi116 running (10m) 15s ago 10m 52.9M 4096M 17.2.0 e1d6a67b021e 6aecbb10256b 2024-06-01T02:04:11.053 INFO:teuthology.orchestra.run.smithi116.stdout:osd.3 smithi116 running (10m) 15s ago 10m 55.0M 4096M 17.2.0 e1d6a67b021e a802290f5290 2024-06-01T02:04:11.053 INFO:teuthology.orchestra.run.smithi116.stdout:osd.4 smithi161 running (10m) 15s ago 10m 56.9M 4096M 17.2.0 e1d6a67b021e 70b3f89b0bd9 2024-06-01T02:04:11.053 INFO:teuthology.orchestra.run.smithi116.stdout:osd.5 smithi161 running (9m) 15s ago 9m 51.3M 4096M 17.2.0 e1d6a67b021e 68358b78a25d 2024-06-01T02:04:11.053 INFO:teuthology.orchestra.run.smithi116.stdout:osd.6 smithi161 running (9m) 15s ago 9m 51.3M 4096M 17.2.0 e1d6a67b021e 50abeb0551e7 2024-06-01T02:04:11.053 INFO:teuthology.orchestra.run.smithi116.stdout:osd.7 smithi161 running (8m) 15s ago 8m 54.4M 4096M 17.2.0 e1d6a67b021e 6004e6983735 2024-06-01T02:04:11.053 INFO:teuthology.orchestra.run.smithi116.stdout:prometheus.a smithi161 *:9095 running (48s) 15s ago 8m 62.7M - 2.33.4 514e6a882f6e e977f6ca1d73 2024-06-01T02:04:11.053 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi116.vzjrxj smithi116 *:8000 running (6m) 15s ago 6m 75.2M - 17.2.0 e1d6a67b021e df4fe3c1dbc3 2024-06-01T02:04:11.053 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi161.ashqxz smithi161 *:8000 running (6m) 15s ago 6m 76.3M - 17.2.0 e1d6a67b021e 3eecc1f3e2d3 2024-06-01T02:04:11.053 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi116.hokbmf smithi116 *:80 running (6m) 15s ago 6m 75.6M - 17.2.0 e1d6a67b021e 10950ed75dbd 2024-06-01T02:04:11.053 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi161.axjlzw smithi161 *:80 running (6m) 15s ago 5m 74.7M - 17.2.0 e1d6a67b021e 0b5d5ce47772 2024-06-01T02:04:11.488 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:04:11.488 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": { 2024-06-01T02:04:11.489 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 3 2024-06-01T02:04:11.489 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:04:11.489 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": { 2024-06-01T02:04:11.489 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-01T02:04:11.489 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:04:11.489 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": { 2024-06-01T02:04:11.489 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2024-06-01T02:04:11.489 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:04:11.489 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": {}, 2024-06-01T02:04:11.489 INFO:teuthology.orchestra.run.smithi116.stdout: "rgw": { 2024-06-01T02:04:11.489 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-06-01T02:04:11.490 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:04:11.490 INFO:teuthology.orchestra.run.smithi116.stdout: "overall": { 2024-06-01T02:04:11.490 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 15, 2024-06-01T02:04:11.490 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-01T02:04:11.490 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T02:04:11.490 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:04:11.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:11.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:11.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:11.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='client.25516 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:04:11.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:11.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:11.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:11.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:11.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:11.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:11.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:11.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:11.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:11.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:11.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:11.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:11.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:11.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:11.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:11.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:11.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:11.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:11.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='client.25522 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:04:11.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:11.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:11.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:11.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:11.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:11.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:11.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:11.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:11.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:11.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:11.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='client.25516 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:04:11.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:11.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:11.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:11.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:11.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:11.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:11.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:11.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:11.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:11.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:11.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:11.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:11.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:11.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:11.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:11.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:11.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:11.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:11.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='client.25522 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:04:11.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:11.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:11.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:11.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:11.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:11.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:11.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:11.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:11.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:11.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:11.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:11.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:11.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:11.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:11.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:11.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:11.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:11.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:11.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='client.15630 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:04:11.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:11.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:11.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:11.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:11.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:11.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:11.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:11.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:11.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:11.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:11.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:11.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:11.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:11.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:11.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:11.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:11.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:11.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:11.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:11.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:11.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:11.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:11.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:11.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='client.15630 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:04:11.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:11.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:11.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:11.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:11.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:11.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:11.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:11.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:11.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:11.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:11.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:11.555 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:11 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:11.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:11.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:11.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:11.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='client.25516 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:04:11.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:11.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:11.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:11.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:11.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:11.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:11.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:11.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:11.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:11.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:11.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:11.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:11.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:11.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:11.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:11.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:11.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:11.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:11.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='client.25522 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:04:11.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:11.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:11.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:11.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:11.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:11.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:11.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:11.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:11.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:11.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:11.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:11.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:11.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:11.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:11.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:11.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:11.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:11.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:11.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='client.15630 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:04:11.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:11.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:11.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:11.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:11.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:11.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:11.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:11.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:11.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:11.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:11.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:11.701 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:11 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:11.855 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:04:11.855 INFO:teuthology.orchestra.run.smithi116.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-01T02:04:11.855 INFO:teuthology.orchestra.run.smithi116.stdout: "in_progress": true, 2024-06-01T02:04:11.855 INFO:teuthology.orchestra.run.smithi116.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-01T02:04:11.855 INFO:teuthology.orchestra.run.smithi116.stdout: "services_complete": [], 2024-06-01T02:04:11.855 INFO:teuthology.orchestra.run.smithi116.stdout: "progress": "", 2024-06-01T02:04:11.855 INFO:teuthology.orchestra.run.smithi116.stdout: "message": "Currently upgrading mgr daemons", 2024-06-01T02:04:11.855 INFO:teuthology.orchestra.run.smithi116.stdout: "is_paused": false 2024-06-01T02:04:11.855 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:04:12.295 INFO:teuthology.orchestra.run.smithi116.stdout:HEALTH_OK 2024-06-01T02:04:12.513 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:12.514 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:12.514 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: pgmap v186: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:12.514 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:12.514 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:12.514 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:12.514 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:12.514 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1224792930' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:12.514 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:12.514 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:12.514 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:12.514 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:12.514 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:12.514 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:12.515 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:12.515 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:12.515 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:12.515 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:12.515 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:12.515 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:12.515 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:12.515 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:12.515 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:12.515 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:12.515 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:12.515 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:12.515 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='client.15642 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:04:12.515 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:12.515 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:12.516 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:12.516 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:12.516 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:12.516 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: pgmap v186: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:12.516 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:12.516 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:12.516 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:12.516 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:12.516 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1224792930' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:12.516 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:12.516 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:12.517 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:12.517 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:12.517 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:12.517 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:12.517 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:12.517 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:12.517 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:12.517 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:12.517 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:12.517 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:12.517 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:12.517 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:12.517 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:12.518 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:12.518 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:12.518 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:12.518 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='client.15642 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:04:12.518 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:12.518 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:12.518 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:12.518 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:12.518 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:12.518 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:12.519 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:12.519 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:12.519 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:12.519 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:12.519 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:12.519 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:12.519 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:12.519 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:12.519 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:12.519 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:12.519 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:12.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1053799291' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:04:12.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:12.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:12.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:12.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:12.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:12.520 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:12.520 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:12.520 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:12.520 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:12.521 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:12.521 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:12.521 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:12.521 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:12.521 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:12.521 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:12.521 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:12.521 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:12.521 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:12.521 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:12.521 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='client.? 172.21.15.116:0/1053799291' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:04:12.522 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:12.522 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:12.522 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:12.522 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:12.522 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:12 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:12.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:12.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:12.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: pgmap v186: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:12.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:12.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:12.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:12.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:12.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1224792930' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:12.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:12.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:12.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:12.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:12.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:12.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:12.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:12.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:12.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:12.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:12.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:12.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:12.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:12.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:12.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:12.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:12.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:12.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:12.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='client.15642 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:04:12.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:12.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:12.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:12.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:12.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:12.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:12.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:12.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:12.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:12.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:12.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:12.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:12.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:12.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:12.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:12.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:12.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:12.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1053799291' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:04:12.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:12.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:12.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:12.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:12.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:12 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:13.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:13 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:13.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:13 smithi116 ceph-mon[25082]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:13.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:13 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:13.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:13 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:13.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:13 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:13.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:13 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:13.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:13 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:13.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:13 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:13.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:13 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:13.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:13 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:13.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:13 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:13.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:13 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:13.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:13 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-06-01T02:04:13.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:13 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mon ok-to-stop", "ids": ["a"]}]: dispatch 2024-06-01T02:04:13.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:13 smithi116 ceph-mon[25082]: Upgrade: It appears safe to stop mon.a 2024-06-01T02:04:13.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:13 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:13.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:13 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:13.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:13 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:13.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:13 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:13.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:13 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:13.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:13 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:13.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:13 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:13.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:13 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:13.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:13 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:13.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:13 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:13.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:13 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:13.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:13 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:13.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:13 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-06-01T02:04:13.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:13 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mon ok-to-stop", "ids": ["a"]}]: dispatch 2024-06-01T02:04:13.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:13 smithi116 ceph-mon[29958]: Upgrade: It appears safe to stop mon.a 2024-06-01T02:04:13.944 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:04:13 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:04:13.505Z caller=manager.go:609 level=warn component="rule manager" group=nodes msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on(instance) group_left(nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi116\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:04:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:13 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:13 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:13 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:13 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:13 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:13 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:13 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:13 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:13 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:13.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:13 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:13.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:13 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:13.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:13 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:13.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:13 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-06-01T02:04:13.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:13 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "mon ok-to-stop", "ids": ["a"]}]: dispatch 2024-06-01T02:04:13.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:13 smithi161 ceph-mon[26055]: Upgrade: It appears safe to stop mon.a 2024-06-01T02:04:15.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:15 smithi161 ceph-mon[26055]: pgmap v187: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:04:15.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:15 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:15.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:15 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:15.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:15 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:15.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:15 smithi116 ceph-mon[25082]: pgmap v187: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:04:15.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:15 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:15.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:15 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:15.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:15 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:15.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:15 smithi116 ceph-mon[29958]: pgmap v187: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:04:15.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:15 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:15.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:15 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:15.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:15 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:17.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:16 smithi161 ceph-mon[26055]: pgmap v188: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:17.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:16 smithi161 ceph-mon[26055]: Upgrade: Updating mon.a 2024-06-01T02:04:17.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:16 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:17.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:16 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T02:04:17.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:16 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T02:04:17.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:16 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:17.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:16 smithi161 ceph-mon[26055]: Deploying daemon mon.a on smithi116 2024-06-01T02:04:17.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:16 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:17.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:16 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:17.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:16 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:17.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:16 smithi161 ceph-mon[26055]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:04:17.196 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:16 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:04:16] "GET /metrics HTTP/1.1" 200 38245 "" "Prometheus/2.33.4" 2024-06-01T02:04:17.269 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:16 smithi116 ceph-mon[29958]: pgmap v188: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:17.269 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:16 smithi116 ceph-mon[29958]: Upgrade: Updating mon.a 2024-06-01T02:04:17.269 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:16 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:17.269 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:16 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T02:04:17.269 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:16 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T02:04:17.270 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:16 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:17.270 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:16 smithi116 ceph-mon[29958]: Deploying daemon mon.a on smithi116 2024-06-01T02:04:17.270 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:16 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:17.270 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:16 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:17.270 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:16 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:17.270 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:16 smithi116 ceph-mon[29958]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:04:17.270 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:16 smithi116 ceph-mon[25082]: pgmap v188: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:17.270 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:16 smithi116 ceph-mon[25082]: Upgrade: Updating mon.a 2024-06-01T02:04:17.270 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:16 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:17.270 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:16 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T02:04:17.270 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:16 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T02:04:17.271 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:16 smithi116 ceph-mon[25082]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:17.271 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:16 smithi116 ceph-mon[25082]: Deploying daemon mon.a on smithi116 2024-06-01T02:04:17.271 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:16 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:17.271 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:16 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:17.271 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:16 smithi116 ceph-mon[25082]: from='mgr.25082 ' entity='mgr.x' 2024-06-01T02:04:17.271 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:16 smithi116 ceph-mon[25082]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:04:18.694 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:04:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:04:18.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"172.21.15.161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"smithi161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:04:19.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:18 smithi116 ceph-mon[25082]: pgmap v189: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:04:19.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:18 smithi116 ceph-mon[29958]: pgmap v189: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:04:19.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:18 smithi161 ceph-mon[26055]: pgmap v189: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:04:19.337 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:19 smithi116 systemd[1]: Stopping Ceph mon.a for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:04:19.337 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:19 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a[25078]: 2024-06-01T02:04:19.310+0000 7ff706f79700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-01T02:04:19.338 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:19 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a[25078]: 2024-06-01T02:04:19.310+0000 7ff706f79700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-06-01T02:04:19.607 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:19 smithi116 podman[85912]: 2024-06-01 02:04:19.341616183 +0000 UTC m=+0.129707694 container died c821092a538a3a0bb92488e8e7b3648dcbc35769de1be25daffb5cbc9781b79a (image=quay.io/ceph/ceph:v17.2.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a, RELEASE=HEAD, ceph=True, io.openshift.tags=base centos centos-stream, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, name=centos-stream, maintainer=Guillaume Abrioux , io.buildah.version=1.19.8, distribution-scope=public, architecture=x86_64, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, io.k8s.display-name=CentOS Stream 8, vendor=Red Hat, Inc., GIT_BRANCH=HEAD, GIT_CLEAN=True, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, version=8, CEPH_POINT_RELEASE=-17.2.0, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, build-date=2022-05-03T08:36:31.336870, vcs-type=git, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.license_terms=https://centos.org/legal/licensing-policy/, release=754, com.redhat.component=centos-stream-container, io.openshift.expose-services=, GIT_REPO=https://github.com/ceph/ceph-container.git, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image) 2024-06-01T02:04:19.999 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:19 smithi116 podman[85912]: 2024-06-01 02:04:19.71031386 +0000 UTC m=+0.498405368 container cleanup c821092a538a3a0bb92488e8e7b3648dcbc35769de1be25daffb5cbc9781b79a (image=quay.io/ceph/ceph:v17.2.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a, GIT_CLEAN=True, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, CEPH_POINT_RELEASE=-17.2.0, name=centos-stream, ceph=True, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, GIT_REPO=https://github.com/ceph/ceph-container.git, io.k8s.display-name=CentOS Stream 8, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, distribution-scope=public, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, io.openshift.tags=base centos centos-stream, maintainer=Guillaume Abrioux , io.buildah.version=1.19.8, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, vcs-type=git, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., GIT_BRANCH=HEAD, version=8, release=754, io.openshift.expose-services=, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, build-date=2022-05-03T08:36:31.336870, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, RELEASE=HEAD, vendor=Red Hat, Inc., com.redhat.component=centos-stream-container, architecture=x86_64) 2024-06-01T02:04:19.999 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:19 smithi116 bash[85912]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a 2024-06-01T02:04:20.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 podman[85925]: 2024-06-01 02:04:20.003631827 +0000 UTC m=+0.664839626 container remove c821092a538a3a0bb92488e8e7b3648dcbc35769de1be25daffb5cbc9781b79a (image=quay.io/ceph/ceph:v17.2.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a, io.openshift.expose-services=, vendor=Red Hat, Inc., architecture=x86_64, version=8, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, com.redhat.component=centos-stream-container, build-date=2022-05-03T08:36:31.336870, RELEASE=HEAD, io.buildah.version=1.19.8, ceph=True, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, GIT_BRANCH=HEAD, name=centos-stream, maintainer=Guillaume Abrioux , vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, io.openshift.tags=base centos centos-stream, distribution-scope=public, release=754, vcs-type=git, GIT_CLEAN=True, CEPH_POINT_RELEASE=-17.2.0, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, io.k8s.display-name=CentOS Stream 8) 2024-06-01T02:04:20.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mon.a.service: Deactivated successfully. 2024-06-01T02:04:20.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 systemd[1]: Stopped Ceph mon.a for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:04:20.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mon.a.service: Consumed 17.588s CPU time. 2024-06-01T02:04:20.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 systemd[1]: Starting Ceph mon.a for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:04:20.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 podman[86052]: 2024-06-01 02:04:20.470476573 +0000 UTC m=+0.017162615 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:04:20.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 podman[86052]: 2024-06-01 02:04:20.57123468 +0000 UTC m=+0.117920740 container create ff679e84e52755e085e9ad642896e41985ede891ba0f2adf4c3c782ab9bfbedc (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-01T02:04:21.037 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 podman[86052]: 2024-06-01 02:04:20.76610354 +0000 UTC m=+0.312789587 container init ff679e84e52755e085e9ad642896e41985ede891ba0f2adf4c3c782ab9bfbedc (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-01T02:04:21.037 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 podman[86052]: 2024-06-01 02:04:20.769759639 +0000 UTC m=+0.316445685 container start ff679e84e52755e085e9ad642896e41985ede891ba0f2adf4c3c782ab9bfbedc (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-06-01T02:04:21.038 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: set uid:gid to 167:167 (ceph:ceph) 2024-06-01T02:04:21.038 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc), process ceph-mon, pid 2 2024-06-01T02:04:21.038 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: pidfile_write: ignore empty --pid-file 2024-06-01T02:04:21.038 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: load: jerasure load: lrc 2024-06-01T02:04:21.038 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: RocksDB version: 7.9.2 2024-06-01T02:04:21.038 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Git sha 0 2024-06-01T02:04:21.038 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Compile date 2024-05-30 15:42:47 2024-06-01T02:04:21.038 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: DB SUMMARY 2024-06-01T02:04:21.038 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: DB Session ID: EGPNBPQR41YIQNVFS3Y1 2024-06-01T02:04:21.038 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: CURRENT file: CURRENT 2024-06-01T02:04:21.038 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: IDENTITY file: IDENTITY 2024-06-01T02:04:21.038 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1742 Bytes 2024-06-01T02:04:21.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000042.sst 2024-06-01T02:04:21.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000040.log size: 363830 ; 2024-06-01T02:04:21.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.error_if_exists: 0 2024-06-01T02:04:21.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.create_if_missing: 0 2024-06-01T02:04:21.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.paranoid_checks: 1 2024-06-01T02:04:21.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-01T02:04:21.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-01T02:04:21.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-01T02:04:21.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.env: 0x5599d94fbc60 2024-06-01T02:04:21.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.fs: PosixFileSystem 2024-06-01T02:04:21.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.info_log: 0x5599da895b20 2024-06-01T02:04:21.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.max_file_opening_threads: 16 2024-06-01T02:04:21.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.statistics: (nil) 2024-06-01T02:04:21.041 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.use_fsync: 0 2024-06-01T02:04:21.041 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.max_log_file_size: 0 2024-06-01T02:04:21.041 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-01T02:04:21.041 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-01T02:04:21.041 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.keep_log_file_num: 1000 2024-06-01T02:04:21.041 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.recycle_log_file_num: 0 2024-06-01T02:04:21.041 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.allow_fallocate: 1 2024-06-01T02:04:21.041 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.allow_mmap_reads: 0 2024-06-01T02:04:21.041 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.allow_mmap_writes: 0 2024-06-01T02:04:21.041 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.use_direct_reads: 0 2024-06-01T02:04:21.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-01T02:04:21.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.create_missing_column_families: 0 2024-06-01T02:04:21.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.db_log_dir: 2024-06-01T02:04:21.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.wal_dir: 2024-06-01T02:04:21.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-01T02:04:21.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-01T02:04:21.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-01T02:04:21.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-01T02:04:21.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-01T02:04:21.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-01T02:04:21.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.advise_random_on_open: 1 2024-06-01T02:04:21.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.db_write_buffer_size: 0 2024-06-01T02:04:21.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.write_buffer_manager: 0x5599da866820 2024-06-01T02:04:21.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-01T02:04:21.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-01T02:04:21.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-01T02:04:21.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.rate_limiter: (nil) 2024-06-01T02:04:21.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-01T02:04:21.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.wal_recovery_mode: 2 2024-06-01T02:04:21.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.enable_thread_tracking: 0 2024-06-01T02:04:21.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.enable_pipelined_write: 0 2024-06-01T02:04:21.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.unordered_write: 0 2024-06-01T02:04:21.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-01T02:04:21.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-01T02:04:21.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-01T02:04:21.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-01T02:04:21.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.row_cache: None 2024-06-01T02:04:21.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.wal_filter: None 2024-06-01T02:04:21.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-01T02:04:21.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.allow_ingest_behind: 0 2024-06-01T02:04:21.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.two_write_queues: 0 2024-06-01T02:04:21.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.manual_wal_flush: 0 2024-06-01T02:04:21.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.wal_compression: 0 2024-06-01T02:04:21.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.atomic_flush: 0 2024-06-01T02:04:21.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-01T02:04:21.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-01T02:04:21.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-01T02:04:21.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.log_readahead_size: 0 2024-06-01T02:04:21.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-01T02:04:21.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.best_efforts_recovery: 0 2024-06-01T02:04:21.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-01T02:04:21.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-01T02:04:21.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.allow_data_in_errors: 0 2024-06-01T02:04:21.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.db_host_id: __hostname__ 2024-06-01T02:04:21.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-01T02:04:21.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.max_background_jobs: 2 2024-06-01T02:04:21.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.max_background_compactions: -1 2024-06-01T02:04:21.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.max_subcompactions: 1 2024-06-01T02:04:21.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-01T02:04:21.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-01T02:04:21.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-01T02:04:21.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.max_total_wal_size: 0 2024-06-01T02:04:21.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-01T02:04:21.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-01T02:04:21.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-01T02:04:21.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-01T02:04:21.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.max_open_files: -1 2024-06-01T02:04:21.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.bytes_per_sync: 0 2024-06-01T02:04:21.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-01T02:04:21.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-01T02:04:21.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.compaction_readahead_size: 0 2024-06-01T02:04:21.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.max_background_flushes: -1 2024-06-01T02:04:21.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Compression algorithms supported: 2024-06-01T02:04:21.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-01T02:04:21.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: kZSTD supported: 0 2024-06-01T02:04:21.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: kXpressCompression supported: 0 2024-06-01T02:04:21.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: kLZ4HCCompression supported: 1 2024-06-01T02:04:21.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: kZlibCompression supported: 1 2024-06-01T02:04:21.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: kSnappyCompression supported: 1 2024-06-01T02:04:21.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: kLZ4Compression supported: 1 2024-06-01T02:04:21.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: kBZip2Compression supported: 0 2024-06-01T02:04:21.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-01T02:04:21.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-01T02:04:21.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000015 2024-06-01T02:04:21.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-01T02:04:21.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-01T02:04:21.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.merge_operator: 2024-06-01T02:04:21.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.compaction_filter: None 2024-06-01T02:04:21.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.compaction_filter_factory: None 2024-06-01T02:04:21.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.sst_partitioner_factory: None 2024-06-01T02:04:21.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-01T02:04:21.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-01T02:04:21.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5599da895c40) 2024-06-01T02:04:21.048 INFO:journalctl@ceph.mon.a.smithi116.stdout: cache_index_and_filter_blocks: 1 2024-06-01T02:04:21.048 INFO:journalctl@ceph.mon.a.smithi116.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-01T02:04:21.048 INFO:journalctl@ceph.mon.a.smithi116.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-01T02:04:21.048 INFO:journalctl@ceph.mon.a.smithi116.stdout: pin_top_level_index_and_filter: 1 2024-06-01T02:04:21.048 INFO:journalctl@ceph.mon.a.smithi116.stdout: index_type: 0 2024-06-01T02:04:21.048 INFO:journalctl@ceph.mon.a.smithi116.stdout: data_block_index_type: 0 2024-06-01T02:04:21.048 INFO:journalctl@ceph.mon.a.smithi116.stdout: index_shortening: 1 2024-06-01T02:04:21.049 INFO:journalctl@ceph.mon.a.smithi116.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-01T02:04:21.049 INFO:journalctl@ceph.mon.a.smithi116.stdout: checksum: 4 2024-06-01T02:04:21.049 INFO:journalctl@ceph.mon.a.smithi116.stdout: no_block_cache: 0 2024-06-01T02:04:21.049 INFO:journalctl@ceph.mon.a.smithi116.stdout: block_cache: 0x5599da897090 2024-06-01T02:04:21.049 INFO:journalctl@ceph.mon.a.smithi116.stdout: block_cache_name: BinnedLRUCache 2024-06-01T02:04:21.049 INFO:journalctl@ceph.mon.a.smithi116.stdout: block_cache_options: 2024-06-01T02:04:21.049 INFO:journalctl@ceph.mon.a.smithi116.stdout: capacity : 536870912 2024-06-01T02:04:21.049 INFO:journalctl@ceph.mon.a.smithi116.stdout: num_shard_bits : 4 2024-06-01T02:04:21.049 INFO:journalctl@ceph.mon.a.smithi116.stdout: strict_capacity_limit : 0 2024-06-01T02:04:21.049 INFO:journalctl@ceph.mon.a.smithi116.stdout: high_pri_pool_ratio: 0.000 2024-06-01T02:04:21.049 INFO:journalctl@ceph.mon.a.smithi116.stdout: block_cache_compressed: (nil) 2024-06-01T02:04:21.049 INFO:journalctl@ceph.mon.a.smithi116.stdout: persistent_cache: (nil) 2024-06-01T02:04:21.049 INFO:journalctl@ceph.mon.a.smithi116.stdout: block_size: 4096 2024-06-01T02:04:21.049 INFO:journalctl@ceph.mon.a.smithi116.stdout: block_size_deviation: 10 2024-06-01T02:04:21.049 INFO:journalctl@ceph.mon.a.smithi116.stdout: block_restart_interval: 16 2024-06-01T02:04:21.049 INFO:journalctl@ceph.mon.a.smithi116.stdout: index_block_restart_interval: 1 2024-06-01T02:04:21.050 INFO:journalctl@ceph.mon.a.smithi116.stdout: metadata_block_size: 4096 2024-06-01T02:04:21.050 INFO:journalctl@ceph.mon.a.smithi116.stdout: partition_filters: 0 2024-06-01T02:04:21.050 INFO:journalctl@ceph.mon.a.smithi116.stdout: use_delta_encoding: 1 2024-06-01T02:04:21.050 INFO:journalctl@ceph.mon.a.smithi116.stdout: filter_policy: bloomfilter 2024-06-01T02:04:21.050 INFO:journalctl@ceph.mon.a.smithi116.stdout: whole_key_filtering: 1 2024-06-01T02:04:21.050 INFO:journalctl@ceph.mon.a.smithi116.stdout: verify_compression: 0 2024-06-01T02:04:21.050 INFO:journalctl@ceph.mon.a.smithi116.stdout: read_amp_bytes_per_bit: 0 2024-06-01T02:04:21.050 INFO:journalctl@ceph.mon.a.smithi116.stdout: format_version: 5 2024-06-01T02:04:21.050 INFO:journalctl@ceph.mon.a.smithi116.stdout: enable_index_compression: 1 2024-06-01T02:04:21.050 INFO:journalctl@ceph.mon.a.smithi116.stdout: block_align: 0 2024-06-01T02:04:21.050 INFO:journalctl@ceph.mon.a.smithi116.stdout: max_auto_readahead_size: 262144 2024-06-01T02:04:21.050 INFO:journalctl@ceph.mon.a.smithi116.stdout: prepopulate_block_cache: 0 2024-06-01T02:04:21.050 INFO:journalctl@ceph.mon.a.smithi116.stdout: initial_auto_readahead_size: 8192 2024-06-01T02:04:21.051 INFO:journalctl@ceph.mon.a.smithi116.stdout: num_file_reads_for_auto_readahead: 2 2024-06-01T02:04:21.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.write_buffer_size: 33554432 2024-06-01T02:04:21.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.max_write_buffer_number: 2 2024-06-01T02:04:21.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.compression: NoCompression 2024-06-01T02:04:21.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.bottommost_compression: Disabled 2024-06-01T02:04:21.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.prefix_extractor: nullptr 2024-06-01T02:04:21.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-01T02:04:21.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.num_levels: 7 2024-06-01T02:04:21.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-01T02:04:21.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-01T02:04:21.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-01T02:04:21.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-01T02:04:21.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-01T02:04:21.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-01T02:04:21.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-01T02:04:21.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-01T02:04:21.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-01T02:04:21.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-01T02:04:21.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-01T02:04:21.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-01T02:04:21.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-01T02:04:21.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.compression_opts.level: 32767 2024-06-01T02:04:21.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.compression_opts.strategy: 0 2024-06-01T02:04:21.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-01T02:04:21.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-01T02:04:21.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-01T02:04:21.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-01T02:04:21.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.compression_opts.enabled: false 2024-06-01T02:04:21.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-01T02:04:21.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-01T02:04:21.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-01T02:04:21.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-01T02:04:21.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.target_file_size_base: 67108864 2024-06-01T02:04:21.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-01T02:04:21.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-01T02:04:21.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-01T02:04:21.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-01T02:04:21.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-01T02:04:21.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-01T02:04:21.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-01T02:04:21.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-01T02:04:21.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-01T02:04:21.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-01T02:04:21.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-01T02:04:21.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-01T02:04:21.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-01T02:04:21.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-01T02:04:21.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.arena_block_size: 1048576 2024-06-01T02:04:21.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-01T02:04:21.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-01T02:04:21.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.disable_auto_compactions: 0 2024-06-01T02:04:21.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-01T02:04:21.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-01T02:04:21.055 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-01T02:04:21.055 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-01T02:04:21.055 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-01T02:04:21.055 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-01T02:04:21.055 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-01T02:04:21.055 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-01T02:04:21.055 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-01T02:04:21.055 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-01T02:04:21.055 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.table_properties_collectors: 2024-06-01T02:04:21.055 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.inplace_update_support: 0 2024-06-01T02:04:21.055 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-01T02:04:21.055 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-01T02:04:21.055 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-01T02:04:21.055 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-01T02:04:21.056 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.bloom_locality: 0 2024-06-01T02:04:21.056 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.max_successive_merges: 0 2024-06-01T02:04:21.056 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-01T02:04:21.056 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.paranoid_file_checks: 0 2024-06-01T02:04:21.056 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.force_consistency_checks: 1 2024-06-01T02:04:21.056 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.report_bg_io_stats: 0 2024-06-01T02:04:21.056 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.ttl: 2592000 2024-06-01T02:04:21.056 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-01T02:04:21.056 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-01T02:04:21.056 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-01T02:04:21.056 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.enable_blob_files: false 2024-06-01T02:04:21.056 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.min_blob_size: 0 2024-06-01T02:04:21.056 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.blob_file_size: 268435456 2024-06-01T02:04:21.056 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-01T02:04:21.056 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-01T02:04:21.056 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-01T02:04:21.057 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-01T02:04:21.057 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-01T02:04:21.057 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.blob_file_starting_level: 0 2024-06-01T02:04:21.057 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-01T02:04:21.057 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 42.sst 2024-06-01T02:04:21.057 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-06-01T02:04:21.057 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 44, last_sequence is 21064, log_number is 40,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-06-01T02:04:21.057 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 40 2024-06-01T02:04:21.057 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5755b057-1280-4380-8230-b004a6acc7a7 2024-06-01T02:04:21.057 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717207460799931, "job": 1, "event": "recovery_started", "wal_files": [40]} 2024-06-01T02:04:21.057 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #40 mode 2 2024-06-01T02:04:21.057 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717207460801768, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 45, "file_size": 344706, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 21066, "largest_seqno": 21294, "table_properties": {"data_size": 342492, "index_size": 816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 389, "raw_key_size": 3781, "raw_average_key_size": 25, "raw_value_size": 339361, "raw_average_value_size": 2324, "num_data_blocks": 34, "num_entries": 146, "num_filter_entries": 146, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1717207460, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5755b057-1280-4380-8230-b004a6acc7a7", "db_session_id": "EGPNBPQR41YIQNVFS3Y1", "orig_file_number": 45, "seqno_to_time_mapping": "N/A"}} 2024-06-01T02:04:21.057 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717207460801854, "job": 1, "event": "recovery_finished"} 2024-06-01T02:04:21.057 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: [db/version_set.cc:5047] Creating manifest 47 2024-06-01T02:04:21.058 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-06-01T02:04:21.058 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000040.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-01T02:04:21.058 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5599da986000 2024-06-01T02:04:21.058 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: DB pointer 0x5599da970000 2024-06-01T02:04:21.058 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-01T02:04:21.058 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-01T02:04:21.058 INFO:journalctl@ceph.mon.a.smithi116.stdout: ** DB Stats ** 2024-06-01T02:04:21.058 INFO:journalctl@ceph.mon.a.smithi116.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-01T02:04:21.058 INFO:journalctl@ceph.mon.a.smithi116.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-01T02:04:21.058 INFO:journalctl@ceph.mon.a.smithi116.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-01T02:04:21.058 INFO:journalctl@ceph.mon.a.smithi116.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-01T02:04:21.058 INFO:journalctl@ceph.mon.a.smithi116.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-01T02:04:21.058 INFO:journalctl@ceph.mon.a.smithi116.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-01T02:04:21.058 INFO:journalctl@ceph.mon.a.smithi116.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-01T02:04:21.058 INFO:journalctl@ceph.mon.a.smithi116.stdout: 2024-06-01T02:04:21.059 INFO:journalctl@ceph.mon.a.smithi116.stdout: ** Compaction Stats [default] ** 2024-06-01T02:04:21.059 INFO:journalctl@ceph.mon.a.smithi116.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-01T02:04:21.059 INFO:journalctl@ceph.mon.a.smithi116.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-01T02:04:21.059 INFO:journalctl@ceph.mon.a.smithi116.stdout: L0 1/0 336.63 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 264.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-01T02:04:21.059 INFO:journalctl@ceph.mon.a.smithi116.stdout: L6 1/0 10.01 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-06-01T02:04:21.059 INFO:journalctl@ceph.mon.a.smithi116.stdout: Sum 2/0 10.34 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 264.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-01T02:04:21.059 INFO:journalctl@ceph.mon.a.smithi116.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 264.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-01T02:04:21.059 INFO:journalctl@ceph.mon.a.smithi116.stdout: 2024-06-01T02:04:21.059 INFO:journalctl@ceph.mon.a.smithi116.stdout: ** Compaction Stats [default] ** 2024-06-01T02:04:21.059 INFO:journalctl@ceph.mon.a.smithi116.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-01T02:04:21.059 INFO:journalctl@ceph.mon.a.smithi116.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-01T02:04:21.059 INFO:journalctl@ceph.mon.a.smithi116.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 264.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-01T02:04:21.059 INFO:journalctl@ceph.mon.a.smithi116.stdout: 2024-06-01T02:04:21.059 INFO:journalctl@ceph.mon.a.smithi116.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-01T02:04:21.059 INFO:journalctl@ceph.mon.a.smithi116.stdout: 2024-06-01T02:04:21.060 INFO:journalctl@ceph.mon.a.smithi116.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-01T02:04:21.060 INFO:journalctl@ceph.mon.a.smithi116.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-01T02:04:21.060 INFO:journalctl@ceph.mon.a.smithi116.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-01T02:04:21.060 INFO:journalctl@ceph.mon.a.smithi116.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-01T02:04:21.060 INFO:journalctl@ceph.mon.a.smithi116.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-01T02:04:21.060 INFO:journalctl@ceph.mon.a.smithi116.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-01T02:04:21.060 INFO:journalctl@ceph.mon.a.smithi116.stdout: Cumulative compaction: 0.00 GB write, 83.73 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-01T02:04:21.060 INFO:journalctl@ceph.mon.a.smithi116.stdout: Interval compaction: 0.00 GB write, 83.73 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-01T02:04:21.060 INFO:journalctl@ceph.mon.a.smithi116.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-01T02:04:21.060 INFO:journalctl@ceph.mon.a.smithi116.stdout: Block cache BinnedLRUCache@0x5599da897090#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-06-01T02:04:21.061 INFO:journalctl@ceph.mon.a.smithi116.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.44 KB,8.34465e-05%) IndexBlock(1,0.95 KB,0.000181794%) Misc(1,0.00 KB,0%) 2024-06-01T02:04:21.061 INFO:journalctl@ceph.mon.a.smithi116.stdout: 2024-06-01T02:04:21.061 INFO:journalctl@ceph.mon.a.smithi116.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-01T02:04:21.061 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: starting mon.a rank 0 at public addrs [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] at bind addrs [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T02:04:21.061 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: mon.a@-1(???) e3 preinit fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T02:04:21.061 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: mon.a@-1(???).mds e1 new map 2024-06-01T02:04:21.061 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: mon.a@-1(???).mds e1 print_map 2024-06-01T02:04:21.061 INFO:journalctl@ceph.mon.a.smithi116.stdout: e1 2024-06-01T02:04:21.061 INFO:journalctl@ceph.mon.a.smithi116.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-01T02:04:21.061 INFO:journalctl@ceph.mon.a.smithi116.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-01T02:04:21.061 INFO:journalctl@ceph.mon.a.smithi116.stdout: legacy client fscid: -1 2024-06-01T02:04:21.061 INFO:journalctl@ceph.mon.a.smithi116.stdout: 2024-06-01T02:04:21.061 INFO:journalctl@ceph.mon.a.smithi116.stdout: No filesystems configured 2024-06-01T02:04:21.061 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: mon.a@-1(???).osd e113 crush map has features 3314933000854323200, adjusting msgr requires 2024-06-01T02:04:21.062 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: mon.a@-1(???).osd e113 crush map has features 432629239337189376, adjusting msgr requires 2024-06-01T02:04:21.062 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: mon.a@-1(???).osd e113 crush map has features 432629239337189376, adjusting msgr requires 2024-06-01T02:04:21.062 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: mon.a@-1(???).osd e113 crush map has features 432629239337189376, adjusting msgr requires 2024-06-01T02:04:21.062 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 ceph-mon[86071]: mon.a@-1(???).paxosservice(auth 1..24) refresh upgraded, format 0 -> 3 2024-06-01T02:04:21.062 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 bash[86052]: ff679e84e52755e085e9ad642896e41985ede891ba0f2adf4c3c782ab9bfbedc 2024-06-01T02:04:21.062 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:20 smithi116 systemd[1]: Started Ceph mon.a for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:04:23.873 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:04:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:04:23.505Z caller=manager.go:609 level=warn component="rule manager" group=nodes msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on(instance) group_left(nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi116\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:04:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:23 smithi161 ceph-mon[26055]: pgmap v190: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:23 smithi161 ceph-mon[26055]: mon.a calling monitor election 2024-06-01T02:04:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:23 smithi161 ceph-mon[26055]: pgmap v191: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:23 smithi161 ceph-mon[26055]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:04:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:23 smithi161 ceph-mon[26055]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-01T02:04:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:23 smithi161 ceph-mon[26055]: monmap epoch 3 2024-06-01T02:04:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:23 smithi161 ceph-mon[26055]: fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T02:04:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:23 smithi161 ceph-mon[26055]: last_changed 2024-06-01T01:51:54.507771+0000 2024-06-01T02:04:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:23 smithi161 ceph-mon[26055]: created 2024-06-01T01:50:20.260187+0000 2024-06-01T02:04:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:23 smithi161 ceph-mon[26055]: min_mon_release 17 (quincy) 2024-06-01T02:04:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:23 smithi161 ceph-mon[26055]: election_strategy: 1 2024-06-01T02:04:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:23 smithi161 ceph-mon[26055]: 0: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.a 2024-06-01T02:04:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:23 smithi161 ceph-mon[26055]: 1: [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] mon.c 2024-06-01T02:04:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:23 smithi161 ceph-mon[26055]: 2: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-06-01T02:04:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:23 smithi161 ceph-mon[26055]: fsmap 2024-06-01T02:04:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:23 smithi161 ceph-mon[26055]: osdmap e113: 8 total, 8 up, 8 in 2024-06-01T02:04:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:23 smithi161 ceph-mon[26055]: mgrmap e38: x(active, since 77s), standbys: y 2024-06-01T02:04:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:23 smithi161 ceph-mon[26055]: overall HEALTH_OK 2024-06-01T02:04:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:23 smithi161 ceph-mon[26055]: from='mgr.25082 ' entity='' 2024-06-01T02:04:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:23 smithi161 ceph-mon[26055]: mgrmap e39: x(active, since 77s), standbys: y 2024-06-01T02:04:24.196 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ignoring --setuser ceph since I am not root 2024-06-01T02:04:24.196 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ignoring --setgroup ceph since I am not root 2024-06-01T02:04:24.197 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:23 smithi161 ceph-mgr[59641]: -- 172.21.15.161:0/1725083581 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5580cb97a5a0 con 0x5580cb94ec00 2024-06-01T02:04:24.197 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:24 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:04:24.027+0000 7fcf1d659200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-01T02:04:24.207 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[86071]: pgmap v190: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:24.207 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[86071]: mon.a calling monitor election 2024-06-01T02:04:24.207 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[86071]: pgmap v191: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:24.207 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[86071]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:04:24.207 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[86071]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-01T02:04:24.207 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[86071]: monmap epoch 3 2024-06-01T02:04:24.207 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[86071]: fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T02:04:24.207 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[86071]: last_changed 2024-06-01T01:51:54.507771+0000 2024-06-01T02:04:24.208 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[86071]: created 2024-06-01T01:50:20.260187+0000 2024-06-01T02:04:24.208 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[86071]: min_mon_release 17 (quincy) 2024-06-01T02:04:24.208 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[86071]: election_strategy: 1 2024-06-01T02:04:24.208 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[86071]: 0: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.a 2024-06-01T02:04:24.208 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[86071]: 1: [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] mon.c 2024-06-01T02:04:24.208 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[86071]: 2: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-06-01T02:04:24.208 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[86071]: fsmap 2024-06-01T02:04:24.208 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[86071]: osdmap e113: 8 total, 8 up, 8 in 2024-06-01T02:04:24.209 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[86071]: mgrmap e38: x(active, since 77s), standbys: y 2024-06-01T02:04:24.209 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[86071]: overall HEALTH_OK 2024-06-01T02:04:24.209 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[86071]: from='mgr.25082 ' entity='' 2024-06-01T02:04:24.209 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[86071]: mgrmap e39: x(active, since 77s), standbys: y 2024-06-01T02:04:24.210 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: ignoring --setuser ceph since I am not root 2024-06-01T02:04:24.210 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: ignoring --setgroup ceph since I am not root 2024-06-01T02:04:24.210 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mgr[80745]: -- 172.21.15.116:0/2497335029 <== mon.0 v2:172.21.15.116:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bd018c85a0 con 0x55bd0189c800 2024-06-01T02:04:24.211 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:24 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:24.024+0000 7efe98510200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-01T02:04:24.211 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:24 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:24.210+0000 7efe98510200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-01T02:04:24.211 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[29958]: pgmap v190: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:24.211 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[29958]: mon.a calling monitor election 2024-06-01T02:04:24.212 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[29958]: pgmap v191: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:24.212 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[29958]: from='mgr.25082 172.21.15.161:0/2297153397' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:04:24.212 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[29958]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-01T02:04:24.212 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[29958]: monmap epoch 3 2024-06-01T02:04:24.213 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[29958]: fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T02:04:24.213 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[29958]: last_changed 2024-06-01T01:51:54.507771+0000 2024-06-01T02:04:24.213 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[29958]: created 2024-06-01T01:50:20.260187+0000 2024-06-01T02:04:24.213 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[29958]: min_mon_release 17 (quincy) 2024-06-01T02:04:24.213 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[29958]: election_strategy: 1 2024-06-01T02:04:24.214 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[29958]: 0: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.a 2024-06-01T02:04:24.214 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[29958]: 1: [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] mon.c 2024-06-01T02:04:24.214 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[29958]: 2: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-06-01T02:04:24.214 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[29958]: fsmap 2024-06-01T02:04:24.215 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[29958]: osdmap e113: 8 total, 8 up, 8 in 2024-06-01T02:04:24.215 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[29958]: mgrmap e38: x(active, since 77s), standbys: y 2024-06-01T02:04:24.215 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[29958]: overall HEALTH_OK 2024-06-01T02:04:24.215 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[29958]: from='mgr.25082 ' entity='' 2024-06-01T02:04:24.216 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:23 smithi116 ceph-mon[29958]: mgrmap e39: x(active, since 77s), standbys: y 2024-06-01T02:04:24.543 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:24 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:24.315+0000 7efe98510200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-01T02:04:24.623 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:24 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:04:24.212+0000 7fcf1d659200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-01T02:04:24.623 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:24 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:04:24.317+0000 7fcf1d659200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-01T02:04:24.623 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:24 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:04:24.627+0000 7fcf1d659200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-01T02:04:24.944 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:24 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:04:24.713+0000 7fcf1d659200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-01T02:04:25.043 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:24 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:24.625+0000 7efe98510200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-01T02:04:25.043 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:24 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:24.711+0000 7efe98510200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-01T02:04:26.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:25 smithi161 ceph-mon[26055]: mgrmap e40: x(active, since 78s), standbys: y 2024-06-01T02:04:26.195 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:25 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:04:25.906+0000 7fcf1d659200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-01T02:04:26.195 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:26 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:04:26.075+0000 7fcf1d659200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-01T02:04:26.248 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:25 smithi116 ceph-mon[29958]: mgrmap e40: x(active, since 78s), standbys: y 2024-06-01T02:04:26.248 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:25 smithi116 ceph-mon[86071]: mgrmap e40: x(active, since 78s), standbys: y 2024-06-01T02:04:26.248 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:25 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:25.906+0000 7efe98510200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-01T02:04:26.248 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:26 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:26.078+0000 7efe98510200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-01T02:04:26.248 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:26 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:26.251+0000 7efe98510200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-01T02:04:26.547 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:26 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:04:26.250+0000 7fcf1d659200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-01T02:04:26.846 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:26 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:04:26.551+0000 7fcf1d659200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-01T02:04:26.846 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:26 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:04:26.646+0000 7fcf1d659200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-01T02:04:26.846 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:26 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:04:26.850+0000 7fcf1d659200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-01T02:04:26.854 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:26 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:26.550+0000 7efe98510200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-01T02:04:26.854 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:26 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:26.648+0000 7efe98510200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-01T02:04:27.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:27 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:04:27.045+0000 7fcf1d659200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-01T02:04:27.293 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:26 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:26.857+0000 7efe98510200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-01T02:04:27.293 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:27 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:27.054+0000 7efe98510200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-01T02:04:27.944 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:27 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:04:27.650+0000 7fcf1d659200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-01T02:04:28.043 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:27 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:27.662+0000 7efe98510200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-01T02:04:28.376 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:28 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:04:28.076+0000 7fcf1d659200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-01T02:04:28.376 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:28 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:04:28.175+0000 7fcf1d659200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-01T02:04:28.376 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:28 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:04:28.380+0000 7fcf1d659200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-01T02:04:28.398 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:28 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:28.096+0000 7efe98510200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-01T02:04:28.398 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:28 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:28.195+0000 7efe98510200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-01T02:04:28.398 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:28 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:28.401+0000 7efe98510200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-01T02:04:29.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:28 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:04:28.905+0000 7fcf1d659200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-01T02:04:29.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:29 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:04:28.999+0000 7fcf1d659200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-01T02:04:29.291 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:28 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:28.944+0000 7efe98510200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-01T02:04:29.292 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:29 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:29.047+0000 7efe98510200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-01T02:04:29.543 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:29 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:29.295+0000 7efe98510200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-01T02:04:29.543 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:29 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:29.431+0000 7efe98510200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-01T02:04:29.543 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:29 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:29.529+0000 7efe98510200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-01T02:04:29.603 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:29 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:04:29.249+0000 7fcf1d659200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-01T02:04:29.603 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:29 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:04:29.386+0000 7fcf1d659200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-01T02:04:29.603 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:29 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:04:29.485+0000 7fcf1d659200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-01T02:04:29.603 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:29 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:04:29.607+0000 7fcf1d659200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-01T02:04:29.949 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:29 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:29.649+0000 7efe98510200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-01T02:04:29.949 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:29 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:29.953+0000 7efe98510200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-01T02:04:30.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:29 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:04:29.912+0000 7fcf1d659200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-01T02:04:30.798 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:30 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:04:30.520+0000 7fcf1d659200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-01T02:04:30.798 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:30 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:04:30.745+0000 7fcf1d659200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-01T02:04:30.900 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:30.565+0000 7efe98510200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-01T02:04:30.900 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:30.795+0000 7efe98510200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-01T02:04:30.900 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: [01/Jun/2024:02:04:30] ENGINE Bus STARTING 2024-06-01T02:04:30.900 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: CherryPy Checker: 2024-06-01T02:04:30.900 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: The Application mounted at '' has an empty config. 2024-06-01T02:04:30.900 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: 2024-06-01T02:04:30.901 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: [01/Jun/2024:02:04:30] ENGINE Serving on http://:::9283 2024-06-01T02:04:30.901 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y[80741]: [01/Jun/2024:02:04:30] ENGINE Bus STARTED 2024-06-01T02:04:30.901 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[86071]: Active manager daemon x restarted 2024-06-01T02:04:30.901 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[86071]: Activating manager daemon x 2024-06-01T02:04:30.901 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[86071]: osdmap e114: 8 total, 8 up, 8 in 2024-06-01T02:04:30.901 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[86071]: mgrmap e41: x(active, starting, since 0.00815167s), standbys: y 2024-06-01T02:04:30.901 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T02:04:30.901 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T02:04:30.901 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T02:04:30.901 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-01T02:04:30.902 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-01T02:04:30.902 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T02:04:30.902 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T02:04:30.902 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T02:04:30.902 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T02:04:30.902 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T02:04:30.902 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T02:04:30.902 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T02:04:30.902 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T02:04:30.902 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-01T02:04:30.903 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-01T02:04:30.903 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-01T02:04:30.903 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[86071]: Manager daemon x is now available 2024-06-01T02:04:30.903 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[86071]: Standby manager daemon y restarted 2024-06-01T02:04:30.903 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[86071]: Standby manager daemon y started 2024-06-01T02:04:30.903 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[29958]: Active manager daemon x restarted 2024-06-01T02:04:30.903 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[29958]: Activating manager daemon x 2024-06-01T02:04:30.903 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[29958]: osdmap e114: 8 total, 8 up, 8 in 2024-06-01T02:04:30.903 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[29958]: mgrmap e41: x(active, starting, since 0.00815167s), standbys: y 2024-06-01T02:04:30.903 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T02:04:30.903 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T02:04:30.903 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T02:04:30.904 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-01T02:04:30.904 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-01T02:04:30.904 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T02:04:30.904 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T02:04:30.904 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T02:04:30.904 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T02:04:30.904 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T02:04:30.904 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T02:04:30.904 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T02:04:30.905 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T02:04:30.905 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-01T02:04:30.905 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-01T02:04:30.905 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-01T02:04:30.905 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[29958]: Manager daemon x is now available 2024-06-01T02:04:30.905 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[29958]: Standby manager daemon y restarted 2024-06-01T02:04:30.906 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:30 smithi116 ceph-mon[29958]: Standby manager daemon y started 2024-06-01T02:04:31.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:30 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: [01/Jun/2024:02:04:30] ENGINE Bus STARTING 2024-06-01T02:04:31.195 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:31 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: CherryPy Checker: 2024-06-01T02:04:31.195 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:31 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: The Application mounted at '' has an empty config. 2024-06-01T02:04:31.195 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:31 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:04:31.195 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:31 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: [01/Jun/2024:02:04:31] ENGINE Serving on http://:::9283 2024-06-01T02:04:31.195 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:31 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: [01/Jun/2024:02:04:31] ENGINE Bus STARTED 2024-06-01T02:04:31.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:30 smithi161 ceph-mon[26055]: Active manager daemon x restarted 2024-06-01T02:04:31.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:30 smithi161 ceph-mon[26055]: Activating manager daemon x 2024-06-01T02:04:31.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:30 smithi161 ceph-mon[26055]: osdmap e114: 8 total, 8 up, 8 in 2024-06-01T02:04:31.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:30 smithi161 ceph-mon[26055]: mgrmap e41: x(active, starting, since 0.00815167s), standbys: y 2024-06-01T02:04:31.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:30 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T02:04:31.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:30 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T02:04:31.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:30 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T02:04:31.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:30 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-01T02:04:31.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:30 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-01T02:04:31.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:30 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T02:04:31.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:30 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T02:04:31.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:30 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T02:04:31.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:30 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T02:04:31.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:30 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T02:04:31.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:30 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T02:04:31.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:30 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T02:04:31.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:30 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T02:04:31.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:30 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-01T02:04:31.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:30 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-01T02:04:31.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:30 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-01T02:04:31.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:30 smithi161 ceph-mon[26055]: Manager daemon x is now available 2024-06-01T02:04:31.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:30 smithi161 ceph-mon[26055]: Standby manager daemon y restarted 2024-06-01T02:04:31.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:30 smithi161 ceph-mon[26055]: Standby manager daemon y started 2024-06-01T02:04:31.831 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:31 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.116:0/3938975036' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-01T02:04:31.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:31 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.116:0/3938975036' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T02:04:31.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:31 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.116:0/3938975036' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-01T02:04:31.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:31 smithi161 ceph-mon[26055]: from='mgr.? 172.21.15.116:0/3938975036' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T02:04:31.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:31 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:31.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:31 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:04:31.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:31 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:04:31.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:31 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:04:31.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:31 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:04:31.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:31 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:04:31.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:31 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-01T02:04:31.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:31 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-01T02:04:31.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:31 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-01T02:04:31.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:31 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-01T02:04:31.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:31 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:04:31.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:31 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:04:31.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:31 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi116", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:04:31.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:31 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi116", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:04:31.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:31 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:31.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:31 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:31.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:31 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:31.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:31 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:31.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:31 smithi161 ceph-mon[26055]: mgrmap e42: x(active, since 1.01078s), standbys: y 2024-06-01T02:04:31.834 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:31 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:04:31.761+0000 7fcedaabb700 -1 mgr.server handle_report got status from non-daemon mon.a 2024-06-01T02:04:32.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.116:0/3938975036' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-01T02:04:32.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.116:0/3938975036' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T02:04:32.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.116:0/3938975036' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-01T02:04:32.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[29958]: from='mgr.? 172.21.15.116:0/3938975036' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T02:04:32.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:32.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:04:32.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:04:32.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:04:32.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:04:32.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:04:32.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-01T02:04:32.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-01T02:04:32.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-01T02:04:32.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-01T02:04:32.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:04:32.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:04:32.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi116", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:04:32.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi116", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:04:32.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:32.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:32.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:32.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:32.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[29958]: mgrmap e42: x(active, since 1.01078s), standbys: y 2024-06-01T02:04:32.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[86071]: from='mgr.? 172.21.15.116:0/3938975036' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-01T02:04:32.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[86071]: from='mgr.? 172.21.15.116:0/3938975036' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-01T02:04:32.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[86071]: from='mgr.? 172.21.15.116:0/3938975036' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-01T02:04:32.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[86071]: from='mgr.? 172.21.15.116:0/3938975036' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-01T02:04:32.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:32.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:04:32.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi161", "caps": []}]: dispatch 2024-06-01T02:04:32.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:04:32.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:04:32.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:04:32.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-01T02:04:32.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-01T02:04:32.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-01T02:04:32.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-01T02:04:32.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:04:32.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:04:32.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi116", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:04:32.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi116", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:04:32.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:32.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:32.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:32.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:32.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:31 smithi116 ceph-mon[86071]: mgrmap e42: x(active, since 1.01078s), standbys: y 2024-06-01T02:04:33.082 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:32 smithi161 ceph-mon[26055]: Updating smithi116:/etc/ceph/ceph.conf 2024-06-01T02:04:33.082 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:32 smithi161 ceph-mon[26055]: Updating smithi161:/etc/ceph/ceph.conf 2024-06-01T02:04:33.082 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:32 smithi161 ceph-mon[26055]: [01/Jun/2024:02:04:31] ENGINE Bus STARTING 2024-06-01T02:04:33.082 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:32 smithi161 ceph-mon[26055]: [01/Jun/2024:02:04:31] ENGINE Serving on https://172.21.15.161:7150 2024-06-01T02:04:33.082 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:32 smithi161 ceph-mon[26055]: [01/Jun/2024:02:04:31] ENGINE Serving on http://172.21.15.161:8765 2024-06-01T02:04:33.082 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:32 smithi161 ceph-mon[26055]: [01/Jun/2024:02:04:31] ENGINE Bus STARTED 2024-06-01T02:04:33.082 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:32 smithi161 ceph-mon[26055]: Updating smithi116:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-01T02:04:33.082 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:32 smithi161 ceph-mon[26055]: Updating smithi161:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-01T02:04:33.082 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:32 smithi161 ceph-mon[26055]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:04:33.082 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:32 smithi161 ceph-mon[26055]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:04:33.083 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:32 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:33.083 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:32 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:33.083 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:32 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:33.083 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:32 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:33.083 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:32 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:33.083 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:32 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:33.083 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:32 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:33.083 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:32 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:04:33.083 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:32 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:04:33.083 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:32 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:33.222 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[29958]: Updating smithi116:/etc/ceph/ceph.conf 2024-06-01T02:04:33.222 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[29958]: Updating smithi161:/etc/ceph/ceph.conf 2024-06-01T02:04:33.222 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[29958]: [01/Jun/2024:02:04:31] ENGINE Bus STARTING 2024-06-01T02:04:33.222 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[29958]: [01/Jun/2024:02:04:31] ENGINE Serving on https://172.21.15.161:7150 2024-06-01T02:04:33.222 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[29958]: [01/Jun/2024:02:04:31] ENGINE Serving on http://172.21.15.161:8765 2024-06-01T02:04:33.222 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[29958]: [01/Jun/2024:02:04:31] ENGINE Bus STARTED 2024-06-01T02:04:33.223 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[29958]: Updating smithi116:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-01T02:04:33.223 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[29958]: Updating smithi161:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-01T02:04:33.223 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[29958]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:04:33.223 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[29958]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:04:33.223 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:33.223 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:33.223 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:33.223 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:33.223 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:33.223 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:33.223 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:33.223 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:04:33.223 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:04:33.224 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:33.224 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[86071]: Updating smithi116:/etc/ceph/ceph.conf 2024-06-01T02:04:33.224 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[86071]: Updating smithi161:/etc/ceph/ceph.conf 2024-06-01T02:04:33.224 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[86071]: [01/Jun/2024:02:04:31] ENGINE Bus STARTING 2024-06-01T02:04:33.224 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[86071]: [01/Jun/2024:02:04:31] ENGINE Serving on https://172.21.15.161:7150 2024-06-01T02:04:33.224 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[86071]: [01/Jun/2024:02:04:31] ENGINE Serving on http://172.21.15.161:8765 2024-06-01T02:04:33.224 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[86071]: [01/Jun/2024:02:04:31] ENGINE Bus STARTED 2024-06-01T02:04:33.224 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[86071]: Updating smithi116:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-01T02:04:33.224 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[86071]: Updating smithi161:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-01T02:04:33.224 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[86071]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:04:33.224 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[86071]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:04:33.224 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:33.224 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:33.224 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:33.225 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:33.225 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:33.225 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:33.225 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:33.225 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:04:33.225 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi116", "caps": []}]: dispatch 2024-06-01T02:04:33.225 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:32 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:33.694 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:04:33 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:04:33.505Z caller=manager.go:609 level=warn component="rule manager" group=nodes msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on(instance) group_left(nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi116\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:04:33.982 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:33 smithi161 ceph-mon[26055]: Updating smithi116:/etc/ceph/ceph.client.admin.keyring 2024-06-01T02:04:33.982 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:33 smithi161 ceph-mon[26055]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-06-01T02:04:33.982 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:33 smithi161 ceph-mon[26055]: Updating smithi116:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-01T02:04:33.983 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:33 smithi161 ceph-mon[26055]: Updating smithi161:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-01T02:04:33.983 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:33 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:33.983 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:33 smithi161 ceph-mon[26055]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:04:33.983 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:33 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:33.983 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:33 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:33.983 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:33 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:33.983 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:33 smithi161 ceph-mon[26055]: mgrmap e43: x(active, since 2s), standbys: y 2024-06-01T02:04:33.983 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:33 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:33.983 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:33 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:33.983 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:33 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:33.983 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:33 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:33.983 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:33 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:33.983 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:33 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:33.983 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:33 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:34.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[29958]: Updating smithi116:/etc/ceph/ceph.client.admin.keyring 2024-06-01T02:04:34.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[29958]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-06-01T02:04:34.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[29958]: Updating smithi116:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-01T02:04:34.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[29958]: Updating smithi161:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-01T02:04:34.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:34.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[29958]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:04:34.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:34.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:34.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:34.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[29958]: mgrmap e43: x(active, since 2s), standbys: y 2024-06-01T02:04:34.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:34.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:34.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:34.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:34.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:34.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:34.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:34.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[86071]: Updating smithi116:/etc/ceph/ceph.client.admin.keyring 2024-06-01T02:04:34.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[86071]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-06-01T02:04:34.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[86071]: Updating smithi116:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-01T02:04:34.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[86071]: Updating smithi161:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-01T02:04:34.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[86071]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:34.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[86071]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:04:34.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:34.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:34.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:34.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[86071]: mgrmap e43: x(active, since 2s), standbys: y 2024-06-01T02:04:34.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:34.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:34.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:34.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:34.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:34.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:34.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:33 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:34.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:34.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:34.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:34.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:34.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:34.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:34.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:34.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:34.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:34.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:34.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:34.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:34.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:34.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:34.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:34.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:34.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:34.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:34.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:34.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:34.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:34.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:34.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:34.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:34.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:34.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:34.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:34.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:34.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:34.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:34.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:34.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:34.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:34.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:34.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:34.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:34.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:34.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:34 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:35.139 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:35.139 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:35.139 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:35.139 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:35.139 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:35.139 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:35.140 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:35.140 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:35.140 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:35.140 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:35.140 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:35.140 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:35.140 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:35.140 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:35.140 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:35.140 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:35.140 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:35.141 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:35.141 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:35.141 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:35.141 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:35.141 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:35.141 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:35.141 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:35.141 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:35.141 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:35.141 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:35.141 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:35.142 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:35.142 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:35.142 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:35.142 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:35.142 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:35.142 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:35.143 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:35.143 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:35.143 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:35.143 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:35.143 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:35.144 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:35.144 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:35.144 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:35.144 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:35.144 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:35.144 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:35.144 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:35.144 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:35.144 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:35.145 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:35.145 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:35.145 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:35.145 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:35.145 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:35.145 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:35.145 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:35.145 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:35.145 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:35.145 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:35.145 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:35.146 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:35.146 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:35.146 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:35.146 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:35.146 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:35.146 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:35.147 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:35.147 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:35.147 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:35.147 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:35.147 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:35.147 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:35.147 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:35.147 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:35.147 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:35.147 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:35.147 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:34 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:36.144 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:36.144 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:36.145 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:36.145 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:36.145 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:36.145 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:36.145 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:04:36.145 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:36.145 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:36.145 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:36.145 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:36.145 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:36.145 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:36.145 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:36.145 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:36.146 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:36.146 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:36.146 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:36.146 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:36.146 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:36.146 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:36.146 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:36.146 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:36.146 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:36.146 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:36.146 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:36.146 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:36.146 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:36.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:36.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:36.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:36.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:36.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:36.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:36.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:36.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:36.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:36.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: mgrmap e44: x(active, since 4s), standbys: y 2024-06-01T02:04:36.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:36.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:36.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:36.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:36.148 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:36.148 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:36.148 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:36.148 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:36.148 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:36.148 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:35 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:36.191 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:36.192 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:36.192 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:36.192 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:36.192 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:36.192 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:36.192 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:04:36.192 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:36.192 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:36.192 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:36.192 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:36.192 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:36.192 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:36.192 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:36.193 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:36.193 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:36.193 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:36.193 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:36.193 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:36.193 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:36.193 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:36.193 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:36.193 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:36.193 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:36.193 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:36.193 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:36.193 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:36.193 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:36.194 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:36.194 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:36.194 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:36.194 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:36.194 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:36.194 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:36.194 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:36.194 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:36.195 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:36.195 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: mgrmap e44: x(active, since 4s), standbys: y 2024-06-01T02:04:36.195 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:36.195 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:36.195 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:36.195 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:36.195 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:36.195 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:36.195 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:36.195 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:36.195 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:36.195 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:36.196 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:36.196 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:36.196 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:36.196 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:36.196 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:36.196 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:36.196 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:04:36.196 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:36.196 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:36.196 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:36.196 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:36.196 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:36.196 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:36.197 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:36.197 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:36.197 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:36.197 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:36.197 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:36.197 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:36.197 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:36.197 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:36.197 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:36.197 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:36.197 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:36.197 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:36.197 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:36.197 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:36.198 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:36.198 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:36.198 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:36.198 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:36.198 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:36.198 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:36.198 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:36.198 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:36.198 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:36.198 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:36.198 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: mgrmap e44: x(active, since 4s), standbys: y 2024-06-01T02:04:36.199 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:36.199 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:36.199 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:36.199 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:36.199 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:36.199 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:36.199 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:36.199 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:36.199 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:36.199 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:35 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:37.134 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:04:36] "GET /metrics HTTP/1.1" 200 35456 "" "Prometheus/2.33.4" 2024-06-01T02:04:37.134 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:37.134 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:37.134 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:37.134 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:37.135 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:37.135 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:37.135 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:37.135 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:37.135 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:37.135 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:37.135 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:37.135 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:37.135 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:37.135 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:37.135 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:37.135 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:37.135 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:37.135 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:37.136 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:37.136 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:37.136 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:37.136 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:37.136 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:37.136 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:37.136 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:37.136 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:37.136 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:37.136 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:37.136 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:37.136 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:37.136 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:37.137 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:37.137 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:37.137 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-06-01T02:04:37.137 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:36 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon ok-to-stop", "ids": ["a"]}]: dispatch 2024-06-01T02:04:37.228 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:37.228 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:37.228 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:37.228 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:37.228 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:37.228 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:37.229 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:37.229 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:37.229 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:37.229 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:37.229 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:37.229 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:37.229 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:37.229 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:37.229 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:37.229 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:37.229 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:37.229 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:37.229 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:37.229 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:37.230 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:37.230 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:37.230 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:37.230 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:37.230 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:37.230 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:37.230 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:37.230 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:37.230 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:37.230 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:37.230 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:37.230 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:37.230 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:37.231 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-06-01T02:04:37.231 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon ok-to-stop", "ids": ["a"]}]: dispatch 2024-06-01T02:04:37.231 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:37.231 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:37.231 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:37.231 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:37.231 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:37.231 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:37.231 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:37.231 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:37.232 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:37.232 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:37.232 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:37.232 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:37.232 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:37.232 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:37.232 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:37.232 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:37.232 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:37.232 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:37.232 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:37.232 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:37.232 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:37.233 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:37.233 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:37.233 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:37.233 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:37.233 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:37.233 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:37.233 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:37.233 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:37.233 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:37.233 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:37.233 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:37.233 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:37.233 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-06-01T02:04:37.233 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:36 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon ok-to-stop", "ids": ["a"]}]: dispatch 2024-06-01T02:04:38.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:37 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:38.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:37 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:38.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:37 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:38.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:37 smithi161 ceph-mon[26055]: Upgrade: It appears safe to stop mon.a 2024-06-01T02:04:38.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:37 smithi161 ceph-mon[26055]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:04:38.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:37 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:38.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:37 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:38.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:37 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:38.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:37 smithi116 ceph-mon[86071]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:38.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:37 smithi116 ceph-mon[86071]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:38.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:37 smithi116 ceph-mon[86071]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:38.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:37 smithi116 ceph-mon[86071]: Upgrade: It appears safe to stop mon.a 2024-06-01T02:04:38.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:37 smithi116 ceph-mon[86071]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:04:38.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:37 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:38.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:37 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:38.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:37 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:38.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:37 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:38.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:37 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:38.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:37 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:38.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:37 smithi116 ceph-mon[29958]: Upgrade: It appears safe to stop mon.a 2024-06-01T02:04:38.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:37 smithi116 ceph-mon[29958]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:04:38.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:37 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:38.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:37 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:38.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:37 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:38.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:38 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T02:04:38.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:38 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T02:04:38.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:38 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:38.945 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:04:38 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:04:38.660Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"172.21.15.161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"smithi161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:04:38.953 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:38 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T02:04:38.953 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:38 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T02:04:38.953 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:38 smithi116 ceph-mon[86071]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:38.953 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:38 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T02:04:38.953 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:38 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T02:04:38.953 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:38 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:40.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:39 smithi161 ceph-mon[26055]: Upgrade: Updating mon.a 2024-06-01T02:04:40.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:39 smithi161 ceph-mon[26055]: Deploying daemon mon.a on smithi116 2024-06-01T02:04:40.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:39 smithi161 ceph-mon[26055]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-06-01T02:04:40.233 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:39 smithi116 ceph-mon[29958]: Upgrade: Updating mon.a 2024-06-01T02:04:40.233 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:39 smithi116 ceph-mon[29958]: Deploying daemon mon.a on smithi116 2024-06-01T02:04:40.233 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:39 smithi116 ceph-mon[29958]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-06-01T02:04:40.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:39 smithi116 ceph-mon[86071]: Upgrade: Updating mon.a 2024-06-01T02:04:40.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:39 smithi116 ceph-mon[86071]: Deploying daemon mon.a on smithi116 2024-06-01T02:04:40.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:39 smithi116 ceph-mon[86071]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-06-01T02:04:41.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:41 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:41.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:41 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:41.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:41 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:41.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:41 smithi116 ceph-mon[29958]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-06-01T02:04:41.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:41 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:41.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:41 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:41.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:41 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:41.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:41 smithi116 ceph-mon[86071]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-06-01T02:04:41.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:41 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:41.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:41 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:41.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:41 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:41.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:41 smithi161 ceph-mon[26055]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-06-01T02:04:42.634 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:42 smithi116 ceph-mon[86071]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:04:42.634 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:42 smithi116 ceph-mon[29958]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:04:42.665 INFO:teuthology.orchestra.run.smithi116.stdout:true 2024-06-01T02:04:42.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:42 smithi161 ceph-mon[26055]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:04:43.340 INFO:teuthology.orchestra.run.smithi116.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-01T02:04:43.341 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi116 smithi116 running 0s ago 13m - - 2024-06-01T02:04:43.341 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi161 smithi161 running 5s ago 13m - - 2024-06-01T02:04:43.341 INFO:teuthology.orchestra.run.smithi116.stdout:alertmanager.a smithi116 *:9093,9094 running (2m) 0s ago 7m 20.5M - 0.23.0 ba2b418f427c e0c297162ded 2024-06-01T02:04:43.341 INFO:teuthology.orchestra.run.smithi116.stdout:grafana.a smithi161 *:3000 running (2m) 5s ago 7m 46.0M - 8.3.5 dad864ee21e9 00b0a78ee88e 2024-06-01T02:04:43.341 INFO:teuthology.orchestra.run.smithi116.stdout:iscsi.foo.smithi116.gjuxzb smithi116 running (87s) 0s ago 6m 47.6M - 3.5 e1d6a67b021e 0e191fe3e3bb 2024-06-01T02:04:43.341 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.x smithi161 *:8443,9283,8765 running (104s) 5s ago 12m 490M - 19.0.0-2946-g3d89accf b045ac545f82 2d7fbb85e5f7 2024-06-01T02:04:43.341 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.y smithi116 *:8443,9283,8765 running (43s) 0s ago 14m 450M - 19.0.0-2946-g3d89accf b045ac545f82 bc9627c27a57 2024-06-01T02:04:43.341 INFO:teuthology.orchestra.run.smithi116.stdout:mon.a smithi116 running (22s) 0s ago 14m 30.7M 2048M 19.0.0-2946-g3d89accf b045ac545f82 ff679e84e527 2024-06-01T02:04:43.341 INFO:teuthology.orchestra.run.smithi116.stdout:mon.b smithi161 running (12m) 5s ago 12m 59.6M 2048M 17.2.0 e1d6a67b021e 9011986aa6e3 2024-06-01T02:04:43.341 INFO:teuthology.orchestra.run.smithi116.stdout:mon.c smithi116 running (12m) 0s ago 12m 55.9M 2048M 17.2.0 e1d6a67b021e 4a6fe0b16d63 2024-06-01T02:04:43.341 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.a smithi116 *:9100 running (2m) 0s ago 8m 17.6M - 1.3.1 1dbe0e931976 e347748d17bb 2024-06-01T02:04:43.341 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.b smithi161 *:9100 running (2m) 5s ago 8m 17.2M - 1.3.1 1dbe0e931976 6e66182f5a1a 2024-06-01T02:04:43.341 INFO:teuthology.orchestra.run.smithi116.stdout:osd.0 smithi116 running (12m) 0s ago 12m 55.0M 4096M 17.2.0 e1d6a67b021e 0cc929b0ecad 2024-06-01T02:04:43.342 INFO:teuthology.orchestra.run.smithi116.stdout:osd.1 smithi116 running (11m) 0s ago 11m 56.5M 4096M 17.2.0 e1d6a67b021e 6ff7fc5abe58 2024-06-01T02:04:43.342 INFO:teuthology.orchestra.run.smithi116.stdout:osd.2 smithi116 running (11m) 0s ago 11m 53.2M 4096M 17.2.0 e1d6a67b021e 6aecbb10256b 2024-06-01T02:04:43.342 INFO:teuthology.orchestra.run.smithi116.stdout:osd.3 smithi116 running (11m) 0s ago 10m 55.5M 4096M 17.2.0 e1d6a67b021e a802290f5290 2024-06-01T02:04:43.342 INFO:teuthology.orchestra.run.smithi116.stdout:osd.4 smithi161 running (10m) 5s ago 10m 57.4M 4096M 17.2.0 e1d6a67b021e 70b3f89b0bd9 2024-06-01T02:04:43.342 INFO:teuthology.orchestra.run.smithi116.stdout:osd.5 smithi161 running (10m) 5s ago 10m 51.7M 4096M 17.2.0 e1d6a67b021e 68358b78a25d 2024-06-01T02:04:43.342 INFO:teuthology.orchestra.run.smithi116.stdout:osd.6 smithi161 running (9m) 5s ago 9m 52.0M 4096M 17.2.0 e1d6a67b021e 50abeb0551e7 2024-06-01T02:04:43.342 INFO:teuthology.orchestra.run.smithi116.stdout:osd.7 smithi161 running (9m) 5s ago 9m 55.0M 4096M 17.2.0 e1d6a67b021e 6004e6983735 2024-06-01T02:04:43.342 INFO:teuthology.orchestra.run.smithi116.stdout:prometheus.a smithi161 *:9095 running (80s) 5s ago 8m 62.9M - 2.33.4 514e6a882f6e e977f6ca1d73 2024-06-01T02:04:43.342 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi116.vzjrxj smithi116 *:8000 running (6m) 0s ago 6m 75.7M - 17.2.0 e1d6a67b021e df4fe3c1dbc3 2024-06-01T02:04:43.342 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi161.ashqxz smithi161 *:8000 running (6m) 5s ago 6m 76.7M - 17.2.0 e1d6a67b021e 3eecc1f3e2d3 2024-06-01T02:04:43.342 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi116.hokbmf smithi116 *:80 running (6m) 0s ago 6m 76.1M - 17.2.0 e1d6a67b021e 10950ed75dbd 2024-06-01T02:04:43.342 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi161.axjlzw smithi161 *:80 running (6m) 5s ago 6m 75.2M - 17.2.0 e1d6a67b021e 0b5d5ce47772 2024-06-01T02:04:43.631 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:43 smithi116 ceph-mon[29958]: from='client.25579 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:04:43.631 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:43 smithi116 ceph-mon[29958]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-06-01T02:04:43.631 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:43 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:43.631 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:43 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:43.631 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:43 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:43.631 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:43 smithi116 ceph-mon[86071]: from='client.25579 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:04:43.632 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:43 smithi116 ceph-mon[86071]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-06-01T02:04:43.632 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:43 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:43.632 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:43 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:43.632 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:43 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:43.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:43 smithi161 ceph-mon[26055]: from='client.25579 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:04:43.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:43 smithi161 ceph-mon[26055]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-06-01T02:04:43.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:43 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:43.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:43 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:43.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:43 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:43.695 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:04:43 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:04:43.505Z caller=manager.go:609 level=warn component="rule manager" group=nodes msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on(instance) group_left(nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi116\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:04:43.751 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:04:43.752 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": { 2024-06-01T02:04:43.752 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 2, 2024-06-01T02:04:43.752 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 1 2024-06-01T02:04:43.752 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:04:43.752 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": { 2024-06-01T02:04:43.752 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-01T02:04:43.752 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:04:43.752 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": { 2024-06-01T02:04:43.752 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2024-06-01T02:04:43.752 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:04:43.753 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": {}, 2024-06-01T02:04:43.753 INFO:teuthology.orchestra.run.smithi116.stdout: "rgw": { 2024-06-01T02:04:43.753 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-06-01T02:04:43.753 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:04:43.753 INFO:teuthology.orchestra.run.smithi116.stdout: "overall": { 2024-06-01T02:04:43.753 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 14, 2024-06-01T02:04:43.753 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 3 2024-06-01T02:04:43.753 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T02:04:43.753 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:04:44.100 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:04:44.100 INFO:teuthology.orchestra.run.smithi116.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-01T02:04:44.100 INFO:teuthology.orchestra.run.smithi116.stdout: "in_progress": true, 2024-06-01T02:04:44.100 INFO:teuthology.orchestra.run.smithi116.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-01T02:04:44.100 INFO:teuthology.orchestra.run.smithi116.stdout: "services_complete": [ 2024-06-01T02:04:44.100 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr" 2024-06-01T02:04:44.100 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-06-01T02:04:44.100 INFO:teuthology.orchestra.run.smithi116.stdout: "progress": "3/23 daemons upgraded", 2024-06-01T02:04:44.101 INFO:teuthology.orchestra.run.smithi116.stdout: "message": "Currently upgrading mon daemons", 2024-06-01T02:04:44.101 INFO:teuthology.orchestra.run.smithi116.stdout: "is_paused": false 2024-06-01T02:04:44.101 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:04:44.523 INFO:teuthology.orchestra.run.smithi116.stdout:HEALTH_OK 2024-06-01T02:04:44.678 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:44 smithi116 systemd[1]: Stopping Ceph mon.a for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:04:44.678 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:44 smithi116 ceph-mon[86071]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:04:44.678 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:44 smithi116 ceph-mon[86071]: from='client.25591 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:04:44.678 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:44 smithi116 ceph-mon[86071]: from='client.? 172.21.15.116:0/1564182468' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:44.678 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:44 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:44.678 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:44 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:44.678 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:44 smithi116 ceph-mon[86071]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:44.678 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:44 smithi116 ceph-mon[29958]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:04:44.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:44 smithi116 ceph-mon[29958]: from='client.25591 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:04:44.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:44 smithi116 ceph-mon[29958]: from='client.? 172.21.15.116:0/1564182468' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:44.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:44 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:44.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:44 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:44.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:44 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:44.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:44 smithi161 ceph-mon[26055]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:04:44.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:44 smithi161 ceph-mon[26055]: from='client.25591 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:04:44.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:44 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1564182468' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:44.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:44 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:44.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:44 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:44.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:44 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:44.995 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:44 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a[86067]: 2024-06-01T02:04:44.680+0000 7f05fb92b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-01T02:04:44.995 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:44 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a[86067]: 2024-06-01T02:04:44.680+0000 7f05fb92b700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-06-01T02:04:44.995 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:44 smithi116 podman[89338]: 2024-06-01 02:04:44.70231863 +0000 UTC m=+0.244643016 container died ff679e84e52755e085e9ad642896e41985ede891ba0f2adf4c3c782ab9bfbedc (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-06-01T02:04:45.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:45 smithi116 podman[89338]: 2024-06-01 02:04:45.215416962 +0000 UTC m=+0.757741373 container cleanup ff679e84e52755e085e9ad642896e41985ede891ba0f2adf4c3c782ab9bfbedc (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a, ceph=True, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528) 2024-06-01T02:04:45.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:45 smithi116 bash[89338]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a 2024-06-01T02:04:45.696 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:45 smithi116 podman[89356]: 2024-06-01 02:04:45.441803848 +0000 UTC m=+0.731199217 container remove ff679e84e52755e085e9ad642896e41985ede891ba0f2adf4c3c782ab9bfbedc (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True) 2024-06-01T02:04:45.696 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:45 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mon.a.service: Deactivated successfully. 2024-06-01T02:04:45.696 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:45 smithi116 systemd[1]: Stopped Ceph mon.a for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:04:45.697 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:45 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mon.a.service: Consumed 1.512s CPU time. 2024-06-01T02:04:46.021 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:45 smithi116 systemd[1]: Starting Ceph mon.a for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:04:46.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 podman[89483]: 2024-06-01 02:04:45.928367056 +0000 UTC m=+0.017226284 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:04:46.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 podman[89483]: 2024-06-01 02:04:46.034613168 +0000 UTC m=+0.123472372 container create b5e9b6f873ff4557c245c9d8120ac3cf97f27788951a06beb9a5d8a39ee6374b (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-01T02:04:46.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 podman[89483]: 2024-06-01 02:04:46.228664128 +0000 UTC m=+0.317523349 container init b5e9b6f873ff4557c245c9d8120ac3cf97f27788951a06beb9a5d8a39ee6374b (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-06-01T02:04:46.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 podman[89483]: 2024-06-01 02:04:46.231877764 +0000 UTC m=+0.320736997 container start b5e9b6f873ff4557c245c9d8120ac3cf97f27788951a06beb9a5d8a39ee6374b (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc) 2024-06-01T02:04:46.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: set uid:gid to 167:167 (ceph:ceph) 2024-06-01T02:04:46.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc), process ceph-mon, pid 2 2024-06-01T02:04:46.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: pidfile_write: ignore empty --pid-file 2024-06-01T02:04:46.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: load: jerasure load: lrc 2024-06-01T02:04:46.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: RocksDB version: 7.9.2 2024-06-01T02:04:46.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Git sha 0 2024-06-01T02:04:46.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Compile date 2024-05-30 15:42:47 2024-06-01T02:04:46.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: DB SUMMARY 2024-06-01T02:04:46.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: DB Session ID: YGLRRSDL867L3V48J169 2024-06-01T02:04:46.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: CURRENT file: CURRENT 2024-06-01T02:04:46.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: IDENTITY file: IDENTITY 2024-06-01T02:04:46.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: MANIFEST file: MANIFEST-000047 size: 523 Bytes 2024-06-01T02:04:46.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000052.sst 2024-06-01T02:04:46.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000050.log size: 560515 ; 2024-06-01T02:04:46.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.error_if_exists: 0 2024-06-01T02:04:46.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.create_if_missing: 0 2024-06-01T02:04:46.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.paranoid_checks: 1 2024-06-01T02:04:46.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-01T02:04:46.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-01T02:04:46.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-01T02:04:46.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.env: 0x559bf995cc60 2024-06-01T02:04:46.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.fs: PosixFileSystem 2024-06-01T02:04:46.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.info_log: 0x559bfbe37b20 2024-06-01T02:04:46.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.max_file_opening_threads: 16 2024-06-01T02:04:46.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.statistics: (nil) 2024-06-01T02:04:46.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.use_fsync: 0 2024-06-01T02:04:46.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.max_log_file_size: 0 2024-06-01T02:04:46.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-01T02:04:46.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-01T02:04:46.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.keep_log_file_num: 1000 2024-06-01T02:04:46.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.recycle_log_file_num: 0 2024-06-01T02:04:46.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.allow_fallocate: 1 2024-06-01T02:04:46.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.allow_mmap_reads: 0 2024-06-01T02:04:46.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.allow_mmap_writes: 0 2024-06-01T02:04:46.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.use_direct_reads: 0 2024-06-01T02:04:46.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-01T02:04:46.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.create_missing_column_families: 0 2024-06-01T02:04:46.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.db_log_dir: 2024-06-01T02:04:46.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.wal_dir: 2024-06-01T02:04:46.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-01T02:04:46.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-01T02:04:46.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-01T02:04:46.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-01T02:04:46.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-01T02:04:46.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-01T02:04:46.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.advise_random_on_open: 1 2024-06-01T02:04:46.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.db_write_buffer_size: 0 2024-06-01T02:04:46.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.write_buffer_manager: 0x559bfbe08820 2024-06-01T02:04:46.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-01T02:04:46.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-01T02:04:46.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-01T02:04:46.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.rate_limiter: (nil) 2024-06-01T02:04:46.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-01T02:04:46.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.wal_recovery_mode: 2 2024-06-01T02:04:46.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.enable_thread_tracking: 0 2024-06-01T02:04:46.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.enable_pipelined_write: 0 2024-06-01T02:04:46.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.unordered_write: 0 2024-06-01T02:04:46.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-01T02:04:46.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-01T02:04:46.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-01T02:04:46.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-01T02:04:46.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.row_cache: None 2024-06-01T02:04:46.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.wal_filter: None 2024-06-01T02:04:46.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-01T02:04:46.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.allow_ingest_behind: 0 2024-06-01T02:04:46.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.two_write_queues: 0 2024-06-01T02:04:46.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.manual_wal_flush: 0 2024-06-01T02:04:46.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.wal_compression: 0 2024-06-01T02:04:46.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.atomic_flush: 0 2024-06-01T02:04:46.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-01T02:04:46.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-01T02:04:46.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-01T02:04:46.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.log_readahead_size: 0 2024-06-01T02:04:46.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-01T02:04:46.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.best_efforts_recovery: 0 2024-06-01T02:04:46.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-01T02:04:46.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-01T02:04:46.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.allow_data_in_errors: 0 2024-06-01T02:04:46.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.db_host_id: __hostname__ 2024-06-01T02:04:46.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-01T02:04:46.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.max_background_jobs: 2 2024-06-01T02:04:46.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.max_background_compactions: -1 2024-06-01T02:04:46.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.max_subcompactions: 1 2024-06-01T02:04:46.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-01T02:04:46.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-01T02:04:46.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-01T02:04:46.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.max_total_wal_size: 0 2024-06-01T02:04:46.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-01T02:04:46.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-01T02:04:46.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-01T02:04:46.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-01T02:04:46.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.max_open_files: -1 2024-06-01T02:04:46.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.bytes_per_sync: 0 2024-06-01T02:04:46.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-01T02:04:46.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-01T02:04:46.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.compaction_readahead_size: 0 2024-06-01T02:04:46.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.max_background_flushes: -1 2024-06-01T02:04:46.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Compression algorithms supported: 2024-06-01T02:04:46.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-01T02:04:46.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: kZSTD supported: 0 2024-06-01T02:04:46.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: kXpressCompression supported: 0 2024-06-01T02:04:46.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: kLZ4HCCompression supported: 1 2024-06-01T02:04:46.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: kZlibCompression supported: 1 2024-06-01T02:04:46.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: kSnappyCompression supported: 1 2024-06-01T02:04:46.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: kLZ4Compression supported: 1 2024-06-01T02:04:46.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: kBZip2Compression supported: 0 2024-06-01T02:04:46.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-01T02:04:46.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-01T02:04:46.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000047 2024-06-01T02:04:46.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-01T02:04:46.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-01T02:04:46.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.merge_operator: 2024-06-01T02:04:46.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.compaction_filter: None 2024-06-01T02:04:46.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.compaction_filter_factory: None 2024-06-01T02:04:46.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.sst_partitioner_factory: None 2024-06-01T02:04:46.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-01T02:04:46.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-01T02:04:46.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559bfbe37c40) 2024-06-01T02:04:46.304 INFO:journalctl@ceph.mon.a.smithi116.stdout: cache_index_and_filter_blocks: 1 2024-06-01T02:04:46.304 INFO:journalctl@ceph.mon.a.smithi116.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-01T02:04:46.304 INFO:journalctl@ceph.mon.a.smithi116.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-01T02:04:46.304 INFO:journalctl@ceph.mon.a.smithi116.stdout: pin_top_level_index_and_filter: 1 2024-06-01T02:04:46.304 INFO:journalctl@ceph.mon.a.smithi116.stdout: index_type: 0 2024-06-01T02:04:46.304 INFO:journalctl@ceph.mon.a.smithi116.stdout: data_block_index_type: 0 2024-06-01T02:04:46.304 INFO:journalctl@ceph.mon.a.smithi116.stdout: index_shortening: 1 2024-06-01T02:04:46.305 INFO:journalctl@ceph.mon.a.smithi116.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-01T02:04:46.305 INFO:journalctl@ceph.mon.a.smithi116.stdout: checksum: 4 2024-06-01T02:04:46.305 INFO:journalctl@ceph.mon.a.smithi116.stdout: no_block_cache: 0 2024-06-01T02:04:46.305 INFO:journalctl@ceph.mon.a.smithi116.stdout: block_cache: 0x559bfbe39090 2024-06-01T02:04:46.305 INFO:journalctl@ceph.mon.a.smithi116.stdout: block_cache_name: BinnedLRUCache 2024-06-01T02:04:46.305 INFO:journalctl@ceph.mon.a.smithi116.stdout: block_cache_options: 2024-06-01T02:04:46.305 INFO:journalctl@ceph.mon.a.smithi116.stdout: capacity : 536870912 2024-06-01T02:04:46.305 INFO:journalctl@ceph.mon.a.smithi116.stdout: num_shard_bits : 4 2024-06-01T02:04:46.305 INFO:journalctl@ceph.mon.a.smithi116.stdout: strict_capacity_limit : 0 2024-06-01T02:04:46.305 INFO:journalctl@ceph.mon.a.smithi116.stdout: high_pri_pool_ratio: 0.000 2024-06-01T02:04:46.305 INFO:journalctl@ceph.mon.a.smithi116.stdout: block_cache_compressed: (nil) 2024-06-01T02:04:46.305 INFO:journalctl@ceph.mon.a.smithi116.stdout: persistent_cache: (nil) 2024-06-01T02:04:46.305 INFO:journalctl@ceph.mon.a.smithi116.stdout: block_size: 4096 2024-06-01T02:04:46.305 INFO:journalctl@ceph.mon.a.smithi116.stdout: block_size_deviation: 10 2024-06-01T02:04:46.306 INFO:journalctl@ceph.mon.a.smithi116.stdout: block_restart_interval: 16 2024-06-01T02:04:46.306 INFO:journalctl@ceph.mon.a.smithi116.stdout: index_block_restart_interval: 1 2024-06-01T02:04:46.306 INFO:journalctl@ceph.mon.a.smithi116.stdout: metadata_block_size: 4096 2024-06-01T02:04:46.306 INFO:journalctl@ceph.mon.a.smithi116.stdout: partition_filters: 0 2024-06-01T02:04:46.306 INFO:journalctl@ceph.mon.a.smithi116.stdout: use_delta_encoding: 1 2024-06-01T02:04:46.306 INFO:journalctl@ceph.mon.a.smithi116.stdout: filter_policy: bloomfilter 2024-06-01T02:04:46.306 INFO:journalctl@ceph.mon.a.smithi116.stdout: whole_key_filtering: 1 2024-06-01T02:04:46.306 INFO:journalctl@ceph.mon.a.smithi116.stdout: verify_compression: 0 2024-06-01T02:04:46.306 INFO:journalctl@ceph.mon.a.smithi116.stdout: read_amp_bytes_per_bit: 0 2024-06-01T02:04:46.306 INFO:journalctl@ceph.mon.a.smithi116.stdout: format_version: 5 2024-06-01T02:04:46.306 INFO:journalctl@ceph.mon.a.smithi116.stdout: enable_index_compression: 1 2024-06-01T02:04:46.307 INFO:journalctl@ceph.mon.a.smithi116.stdout: block_align: 0 2024-06-01T02:04:46.307 INFO:journalctl@ceph.mon.a.smithi116.stdout: max_auto_readahead_size: 262144 2024-06-01T02:04:46.307 INFO:journalctl@ceph.mon.a.smithi116.stdout: prepopulate_block_cache: 0 2024-06-01T02:04:46.307 INFO:journalctl@ceph.mon.a.smithi116.stdout: initial_auto_readahead_size: 8192 2024-06-01T02:04:46.307 INFO:journalctl@ceph.mon.a.smithi116.stdout: num_file_reads_for_auto_readahead: 2 2024-06-01T02:04:46.307 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.write_buffer_size: 33554432 2024-06-01T02:04:46.307 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.max_write_buffer_number: 2 2024-06-01T02:04:46.307 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.compression: NoCompression 2024-06-01T02:04:46.307 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.bottommost_compression: Disabled 2024-06-01T02:04:46.307 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.prefix_extractor: nullptr 2024-06-01T02:04:46.307 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-01T02:04:46.307 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.num_levels: 7 2024-06-01T02:04:46.307 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-01T02:04:46.307 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-01T02:04:46.308 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-01T02:04:46.308 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-01T02:04:46.308 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-01T02:04:46.308 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-01T02:04:46.308 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-01T02:04:46.308 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-01T02:04:46.308 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-01T02:04:46.308 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-01T02:04:46.308 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-01T02:04:46.308 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-01T02:04:46.308 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-01T02:04:46.308 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.compression_opts.level: 32767 2024-06-01T02:04:46.308 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.compression_opts.strategy: 0 2024-06-01T02:04:46.308 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-01T02:04:46.309 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-01T02:04:46.309 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-01T02:04:46.309 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-01T02:04:46.309 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.compression_opts.enabled: false 2024-06-01T02:04:46.309 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-01T02:04:46.309 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-01T02:04:46.309 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-01T02:04:46.309 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-01T02:04:46.309 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.target_file_size_base: 67108864 2024-06-01T02:04:46.309 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-01T02:04:46.309 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-01T02:04:46.309 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-01T02:04:46.309 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-01T02:04:46.309 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-01T02:04:46.310 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-01T02:04:46.310 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-01T02:04:46.310 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-01T02:04:46.310 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-01T02:04:46.310 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-01T02:04:46.310 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-01T02:04:46.310 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-01T02:04:46.310 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-01T02:04:46.310 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-01T02:04:46.310 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.arena_block_size: 1048576 2024-06-01T02:04:46.310 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-01T02:04:46.310 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-01T02:04:46.310 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.disable_auto_compactions: 0 2024-06-01T02:04:46.310 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-01T02:04:46.311 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-01T02:04:46.311 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-01T02:04:46.311 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-01T02:04:46.311 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-01T02:04:46.311 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-01T02:04:46.311 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-01T02:04:46.311 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-01T02:04:46.311 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-01T02:04:46.311 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-01T02:04:46.311 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.table_properties_collectors: 2024-06-01T02:04:46.311 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.inplace_update_support: 0 2024-06-01T02:04:46.311 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-01T02:04:46.311 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-01T02:04:46.311 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-01T02:04:46.311 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-01T02:04:46.312 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.bloom_locality: 0 2024-06-01T02:04:46.312 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.max_successive_merges: 0 2024-06-01T02:04:46.312 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-01T02:04:46.312 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.paranoid_file_checks: 0 2024-06-01T02:04:46.312 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.force_consistency_checks: 1 2024-06-01T02:04:46.312 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.report_bg_io_stats: 0 2024-06-01T02:04:46.312 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.ttl: 2592000 2024-06-01T02:04:46.312 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-01T02:04:46.312 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-01T02:04:46.312 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-01T02:04:46.312 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.enable_blob_files: false 2024-06-01T02:04:46.312 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.min_blob_size: 0 2024-06-01T02:04:46.312 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.blob_file_size: 268435456 2024-06-01T02:04:46.312 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-01T02:04:46.312 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-01T02:04:46.313 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-01T02:04:46.313 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-01T02:04:46.313 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-01T02:04:46.313 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.blob_file_starting_level: 0 2024-06-01T02:04:46.313 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-01T02:04:46.313 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-06-01T02:04:46.313 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000047 succeeded,manifest_file_number is 47, next_file_number is 54, last_sequence is 22098, log_number is 50,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 50 2024-06-01T02:04:46.313 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 50 2024-06-01T02:04:46.313 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5755b057-1280-4380-8230-b004a6acc7a7 2024-06-01T02:04:46.313 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717207486261503, "job": 1, "event": "recovery_started", "wal_files": [50]} 2024-06-01T02:04:46.313 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #50 mode 2 2024-06-01T02:04:46.313 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717207486263794, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 55, "file_size": 504216, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 22099, "largest_seqno": 22435, "table_properties": {"data_size": 501593, "index_size": 1097, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 517, "raw_key_size": 4778, "raw_average_key_size": 24, "raw_value_size": 497619, "raw_average_value_size": 2565, "num_data_blocks": 47, "num_entries": 194, "num_filter_entries": 194, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1717207486, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5755b057-1280-4380-8230-b004a6acc7a7", "db_session_id": "YGLRRSDL867L3V48J169", "orig_file_number": 55, "seqno_to_time_mapping": "N/A"}} 2024-06-01T02:04:46.313 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717207486263875, "job": 1, "event": "recovery_finished"} 2024-06-01T02:04:46.313 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: [db/version_set.cc:5047] Creating manifest 57 2024-06-01T02:04:46.313 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-06-01T02:04:46.314 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000050.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-01T02:04:46.314 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559bfbf28000 2024-06-01T02:04:46.314 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: DB pointer 0x559bfbf12000 2024-06-01T02:04:46.314 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-01T02:04:46.314 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-01T02:04:46.314 INFO:journalctl@ceph.mon.a.smithi116.stdout: ** DB Stats ** 2024-06-01T02:04:46.314 INFO:journalctl@ceph.mon.a.smithi116.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-01T02:04:46.314 INFO:journalctl@ceph.mon.a.smithi116.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-01T02:04:46.314 INFO:journalctl@ceph.mon.a.smithi116.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-01T02:04:46.314 INFO:journalctl@ceph.mon.a.smithi116.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-01T02:04:46.314 INFO:journalctl@ceph.mon.a.smithi116.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-01T02:04:46.314 INFO:journalctl@ceph.mon.a.smithi116.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-01T02:04:46.314 INFO:journalctl@ceph.mon.a.smithi116.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-01T02:04:46.314 INFO:journalctl@ceph.mon.a.smithi116.stdout: 2024-06-01T02:04:46.315 INFO:journalctl@ceph.mon.a.smithi116.stdout: ** Compaction Stats [default] ** 2024-06-01T02:04:46.315 INFO:journalctl@ceph.mon.a.smithi116.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-01T02:04:46.315 INFO:journalctl@ceph.mon.a.smithi116.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-01T02:04:46.315 INFO:journalctl@ceph.mon.a.smithi116.stdout: L0 1/0 492.40 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 344.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-01T02:04:46.315 INFO:journalctl@ceph.mon.a.smithi116.stdout: L6 1/0 11.81 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-06-01T02:04:46.315 INFO:journalctl@ceph.mon.a.smithi116.stdout: Sum 2/0 12.29 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 344.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-01T02:04:46.315 INFO:journalctl@ceph.mon.a.smithi116.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 344.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-01T02:04:46.315 INFO:journalctl@ceph.mon.a.smithi116.stdout: 2024-06-01T02:04:46.315 INFO:journalctl@ceph.mon.a.smithi116.stdout: ** Compaction Stats [default] ** 2024-06-01T02:04:46.315 INFO:journalctl@ceph.mon.a.smithi116.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-01T02:04:46.315 INFO:journalctl@ceph.mon.a.smithi116.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-01T02:04:46.315 INFO:journalctl@ceph.mon.a.smithi116.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 344.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-01T02:04:46.315 INFO:journalctl@ceph.mon.a.smithi116.stdout: 2024-06-01T02:04:46.315 INFO:journalctl@ceph.mon.a.smithi116.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-01T02:04:46.315 INFO:journalctl@ceph.mon.a.smithi116.stdout: 2024-06-01T02:04:46.316 INFO:journalctl@ceph.mon.a.smithi116.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-01T02:04:46.316 INFO:journalctl@ceph.mon.a.smithi116.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-01T02:04:46.316 INFO:journalctl@ceph.mon.a.smithi116.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-01T02:04:46.316 INFO:journalctl@ceph.mon.a.smithi116.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-01T02:04:46.316 INFO:journalctl@ceph.mon.a.smithi116.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-01T02:04:46.316 INFO:journalctl@ceph.mon.a.smithi116.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-01T02:04:46.316 INFO:journalctl@ceph.mon.a.smithi116.stdout: Cumulative compaction: 0.00 GB write, 110.98 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-01T02:04:46.316 INFO:journalctl@ceph.mon.a.smithi116.stdout: Interval compaction: 0.00 GB write, 110.98 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-01T02:04:46.316 INFO:journalctl@ceph.mon.a.smithi116.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-01T02:04:46.316 INFO:journalctl@ceph.mon.a.smithi116.stdout: Block cache BinnedLRUCache@0x559bfbe39090#2 capacity: 512.00 MB usage: 1.80 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-06-01T02:04:46.316 INFO:journalctl@ceph.mon.a.smithi116.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.59 KB,0.000113249%) IndexBlock(1,1.20 KB,0.000229478%) Misc(1,0.00 KB,0%) 2024-06-01T02:04:46.316 INFO:journalctl@ceph.mon.a.smithi116.stdout: 2024-06-01T02:04:46.316 INFO:journalctl@ceph.mon.a.smithi116.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-01T02:04:46.316 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: starting mon.a rank 0 at public addrs [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] at bind addrs [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T02:04:46.316 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: mon.a@-1(???) e3 preinit fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T02:04:46.317 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: mon.a@-1(???).mds e1 new map 2024-06-01T02:04:46.317 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: mon.a@-1(???).mds e1 print_map 2024-06-01T02:04:46.317 INFO:journalctl@ceph.mon.a.smithi116.stdout: e1 2024-06-01T02:04:46.317 INFO:journalctl@ceph.mon.a.smithi116.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-01T02:04:46.317 INFO:journalctl@ceph.mon.a.smithi116.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-01T02:04:46.317 INFO:journalctl@ceph.mon.a.smithi116.stdout: legacy client fscid: -1 2024-06-01T02:04:46.317 INFO:journalctl@ceph.mon.a.smithi116.stdout: 2024-06-01T02:04:46.317 INFO:journalctl@ceph.mon.a.smithi116.stdout: No filesystems configured 2024-06-01T02:04:46.317 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: mon.a@-1(???).osd e114 crush map has features 3314933000854323200, adjusting msgr requires 2024-06-01T02:04:46.317 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: mon.a@-1(???).osd e114 crush map has features 432629239337189376, adjusting msgr requires 2024-06-01T02:04:46.317 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: mon.a@-1(???).osd e114 crush map has features 432629239337189376, adjusting msgr requires 2024-06-01T02:04:46.317 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: mon.a@-1(???).osd e114 crush map has features 432629239337189376, adjusting msgr requires 2024-06-01T02:04:46.317 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 ceph-mon[89502]: mon.a@-1(???).paxosservice(auth 1..25) refresh upgraded, format 0 -> 3 2024-06-01T02:04:46.566 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 bash[89483]: b5e9b6f873ff4557c245c9d8120ac3cf97f27788951a06beb9a5d8a39ee6374b 2024-06-01T02:04:46.567 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:46 smithi116 systemd[1]: Started Ceph mon.a for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:04:47.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:46 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:04:46] "GET /metrics HTTP/1.1" 200 38262 "" "Prometheus/2.33.4" 2024-06-01T02:04:48.945 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:04:48 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:04:48.660Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"172.21.15.161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"smithi161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:04:49.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:49 smithi161 ceph-mon[26055]: from='client.34158 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:04:49.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:49 smithi161 ceph-mon[26055]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-01T02:04:49.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:49 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:04:49.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:49 smithi161 ceph-mon[26055]: mon.a calling monitor election 2024-06-01T02:04:49.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:49 smithi161 ceph-mon[26055]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-01T02:04:49.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:49 smithi161 ceph-mon[26055]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-01T02:04:49.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:49 smithi161 ceph-mon[26055]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-01T02:04:49.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:49 smithi161 ceph-mon[26055]: monmap epoch 3 2024-06-01T02:04:49.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:49 smithi161 ceph-mon[26055]: fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T02:04:49.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:49 smithi161 ceph-mon[26055]: last_changed 2024-06-01T01:51:54.507771+0000 2024-06-01T02:04:49.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:49 smithi161 ceph-mon[26055]: created 2024-06-01T01:50:20.260187+0000 2024-06-01T02:04:49.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:49 smithi161 ceph-mon[26055]: min_mon_release 17 (quincy) 2024-06-01T02:04:49.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:49 smithi161 ceph-mon[26055]: election_strategy: 1 2024-06-01T02:04:49.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:49 smithi161 ceph-mon[26055]: 0: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.a 2024-06-01T02:04:49.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:49 smithi161 ceph-mon[26055]: 1: [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] mon.c 2024-06-01T02:04:49.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:49 smithi161 ceph-mon[26055]: 2: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-06-01T02:04:49.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:49 smithi161 ceph-mon[26055]: fsmap 2024-06-01T02:04:49.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:49 smithi161 ceph-mon[26055]: osdmap e114: 8 total, 8 up, 8 in 2024-06-01T02:04:49.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:49 smithi161 ceph-mon[26055]: mgrmap e44: x(active, since 18s), standbys: y 2024-06-01T02:04:49.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:49 smithi161 ceph-mon[26055]: overall HEALTH_OK 2024-06-01T02:04:49.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:49 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='' 2024-06-01T02:04:49.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:49 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:49.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[89502]: from='client.34158 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:04:49.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[89502]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-01T02:04:49.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:04:49.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[89502]: mon.a calling monitor election 2024-06-01T02:04:49.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[89502]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-01T02:04:49.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[89502]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-01T02:04:49.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[89502]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-01T02:04:49.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[89502]: monmap epoch 3 2024-06-01T02:04:49.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[89502]: fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T02:04:49.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[89502]: last_changed 2024-06-01T01:51:54.507771+0000 2024-06-01T02:04:49.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[89502]: created 2024-06-01T01:50:20.260187+0000 2024-06-01T02:04:49.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[89502]: min_mon_release 17 (quincy) 2024-06-01T02:04:49.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[89502]: election_strategy: 1 2024-06-01T02:04:49.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[89502]: 0: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.a 2024-06-01T02:04:49.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[89502]: 1: [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] mon.c 2024-06-01T02:04:49.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[89502]: 2: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-06-01T02:04:49.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[89502]: fsmap 2024-06-01T02:04:49.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[89502]: osdmap e114: 8 total, 8 up, 8 in 2024-06-01T02:04:49.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[89502]: mgrmap e44: x(active, since 18s), standbys: y 2024-06-01T02:04:49.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[89502]: overall HEALTH_OK 2024-06-01T02:04:49.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='' 2024-06-01T02:04:49.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:49.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[29958]: from='client.34158 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:04:49.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[29958]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-01T02:04:49.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:04:49.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[29958]: mon.a calling monitor election 2024-06-01T02:04:49.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[29958]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-01T02:04:49.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[29958]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-01T02:04:49.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[29958]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-01T02:04:49.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[29958]: monmap epoch 3 2024-06-01T02:04:49.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[29958]: fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T02:04:49.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[29958]: last_changed 2024-06-01T01:51:54.507771+0000 2024-06-01T02:04:49.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[29958]: created 2024-06-01T01:50:20.260187+0000 2024-06-01T02:04:49.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[29958]: min_mon_release 17 (quincy) 2024-06-01T02:04:49.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[29958]: election_strategy: 1 2024-06-01T02:04:49.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[29958]: 0: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.a 2024-06-01T02:04:49.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[29958]: 1: [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] mon.c 2024-06-01T02:04:49.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[29958]: 2: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-06-01T02:04:49.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[29958]: fsmap 2024-06-01T02:04:49.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[29958]: osdmap e114: 8 total, 8 up, 8 in 2024-06-01T02:04:49.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[29958]: mgrmap e44: x(active, since 18s), standbys: y 2024-06-01T02:04:49.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[29958]: overall HEALTH_OK 2024-06-01T02:04:49.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='' 2024-06-01T02:04:49.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:49 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:50.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:50.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:50.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:50.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:50.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:50.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:50.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:50.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:50.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:50.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:50.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:50.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:50.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:50.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:50.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:50.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:50.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:50.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:50.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:50.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:50.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:50.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:50.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:50.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:50.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:50.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:50.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:50.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:50.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:50.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:50.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:50.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:50.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:50.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:50.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:50.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:50.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:50.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:50.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:50.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:50.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:50.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:50.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:50.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:50 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:50.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:50.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:50.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:50.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:50.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:50.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:50.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:50.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:50.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:50.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:50.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:50.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:50.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:50.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:50.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:50.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:50.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:50.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:50.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:50.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:50.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:50.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:50.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:50.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:50.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:50.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:50.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:50.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:50.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:50.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:50.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:50.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:50.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:50.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:50.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:50.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:50.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:50.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:50.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:50.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:50.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:50.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:50.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:50.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:50.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:50.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:50.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:50.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:50.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:50.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:50.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:50.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:50.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:50.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:50.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:50.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:50.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:50.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:50.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:50.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:50.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:50.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:50.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:50.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:50.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:50.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:50.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:50.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:50.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:50.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:50.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:50.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:50.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:50.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:50.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:50.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:50.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:50.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:50.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:50.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:50.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:50.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:50.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:50.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:50.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:50.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:50.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:50.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:51.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:51.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:51.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:51.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:51.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:51.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:51.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:51.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:51.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:51.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:51.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:51.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:51.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:51.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:51.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:51.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:51.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:51.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:51.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:51.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:51.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:51.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:51.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:51.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:51.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:51.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:51.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:51.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:51.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:51.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:51.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:51.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:51 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:51.789 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.790 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.790 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.790 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.790 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.790 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.790 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:51.790 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.790 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:51.790 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:51.790 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.790 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.790 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:51.791 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.791 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:51.791 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:51.791 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.791 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.791 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:51.791 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.791 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:51.791 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:51.791 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.791 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.791 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:51.791 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.791 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:51.792 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:51.792 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.792 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.792 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:51.792 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.792 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.792 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.792 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.792 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.792 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.792 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:51.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:51.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:51.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:51.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:51.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:51.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:51.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:51.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:51.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:51.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:51.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:51.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:51.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:51.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:51.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:51.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:51.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:51.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:51.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:51.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:51.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:51.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:51.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:51.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:51.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:51.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:51.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:51.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:51.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:51.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:51.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:51.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:51.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:51.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:51.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:51.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:51.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:51.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:51.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:51.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:51.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:51.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:51.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:51.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:51.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:51.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:51.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:51.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:51.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:51.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:51.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:51.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:51.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:51 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:52.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:52.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:52.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:52.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:52.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:52.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:52.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:52.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:52.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:52.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:52.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:52.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:52.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:52.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:52.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:52.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:52.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:52.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:52.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:52.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:52.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:52.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:52.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:52.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:52.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:52.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:52.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:52.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:52.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:04:52.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:52.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:52.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:52.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:52.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:52.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:52.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:52.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:52.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:52.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:52.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:52.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:52.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:52.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:52.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:52.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:52.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:52.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:52.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:52.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:52.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:52.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:52.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:52.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:52.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:52.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:52 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:52.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:52.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:52.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:52.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:52.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:52.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:52.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:52.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:52.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:52.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:52.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:52.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:52.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:52.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:52.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:52.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:52.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:52.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:52.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:52.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:52.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:52.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:52.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:52.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:52.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:52.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:52.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:52.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:52.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:04:52.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:52.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:52.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:52.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:52.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:52.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:52.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:52.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:52.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:52.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:52.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:52.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:52.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:52.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:52.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:52.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:52.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:52.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:52.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:52.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:52.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:52.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:52.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:52.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:52.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:52.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:52.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:52.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:52.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:52.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:04:52.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:52.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:52.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:52.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:52.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:52.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:52.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:52.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:52.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:52.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:52.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:52.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:52.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:52.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:52.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:52.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:52.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:52.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:52.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:52.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:52.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:52.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:52.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:52.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:52.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:52.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:52.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:52.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:52.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:52.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:52.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:52.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:52.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:52.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:52.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:52.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:52.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:52.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:52.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:52.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:52.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:52.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:52.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:52.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:52.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:52.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:52.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:52.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:52.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:52.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:52.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:52 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:53.604 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:53.605 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:53.605 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:53.605 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:53.605 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:53.605 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:53.606 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:53.606 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:53.606 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:53.606 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:53.606 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:53.606 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:53.606 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:53.607 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:53.607 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:53.607 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:53.607 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:53.607 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:53.607 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:53.607 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:53.608 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:53.608 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:53.608 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:04:53.608 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:53.608 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:53.608 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:53.608 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:53.609 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:53.609 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:53.609 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:53.609 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:53.609 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:53.610 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:53.610 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:53.610 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:53.610 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:53.610 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:53.610 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:53.611 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:53.611 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:53.613 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:53.613 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:53.613 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:53.613 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:53.613 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:53.613 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:53.614 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:53.614 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:53.614 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:53.614 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:53.614 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:53.614 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:53.614 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:04:53.614 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:53.614 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:53.614 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:53.614 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:53.614 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:53.615 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:53.615 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:53.615 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:53.615 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:53.615 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:53.615 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:53.615 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:53.615 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:53.615 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:53.615 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:53.616 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:53.616 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:53.616 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:53.616 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:53.616 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:53.616 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:53.616 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:53.616 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:53.616 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:53.616 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:53.616 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:53.616 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:53.617 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:53.617 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:53.617 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:53.617 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:53.617 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:53.617 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:53.617 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:53.617 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:53.617 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:53.617 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:53.618 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:53.618 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:53.618 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:53.618 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:53.618 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:53.618 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:53.618 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:53.618 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:53 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:53.624 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:04:53.504Z caller=manager.go:609 level=warn component="rule manager" group=nodes msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on(instance) group_left(nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi116\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:04:53.624 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:53.625 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:53.625 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:53.625 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:53.625 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:53.625 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:53.625 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:53.625 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:53.625 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:53.625 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:53.625 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:53.625 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:53.625 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:53.625 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:53.625 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:53.626 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:53.626 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:53.626 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:53.626 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:53.626 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:53.626 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:53.626 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:53.626 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:04:53.626 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:53.626 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:53.626 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:53.626 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:53.627 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:53.627 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:53.627 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:53.627 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:53.627 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:53.627 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:53.627 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:53.627 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:53.627 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:53.627 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:53.627 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:53.627 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:53.628 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:53.628 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:53.628 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:53.628 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:53.628 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:53.628 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:53.628 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:53.628 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:53.628 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:53.628 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:53 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:54.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:54.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:54.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:54.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:54.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:54.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:54.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:54.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:54.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:54.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:54.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:54.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:54.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:54.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:54.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:54.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:54.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:54.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:54.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:54.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:54.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:54.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:54.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:54.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:54.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:54.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:54.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:54.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:54.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:54.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:54.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:54.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:54.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:54.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:54.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:54.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:54.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:54.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:54.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:54.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:54.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:54.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:54.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:54.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:54.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:54.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:54.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:54.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:54.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:54.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:54.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:54.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:54.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:54.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:54.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:54.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:54.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:54.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:54.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:54.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:54.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:54.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:54.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:54.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:54.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:54.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:54.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:54.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:54.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:54.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:54.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:54.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:54.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:54.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:54.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:54.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:54.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:54.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:54.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:54.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:54.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:54.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:54.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:54.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:54.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:54.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:54.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:54.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:54.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:54.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:54.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:54.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:54.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:54.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:54.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:54.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:54.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:54.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:54 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:54.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:54.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:54.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:54.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:54.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:54.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:54.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:54.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:54.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:54.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:54.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:54.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:54.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:54.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:54.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:54.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:54.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:54.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:54.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:54.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:54.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:54.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:54.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:54.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:54.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:54.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:54.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:54.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:54.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:54.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:54.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:54.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:54.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:54.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:54.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:54.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:54.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:54.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:54.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:54.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:54.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:54.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:54.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:54.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:54.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:54.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:54.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:54.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:54.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:54 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:55.617 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:55.617 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:55.617 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:55.617 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:55.617 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:55.617 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:55.617 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:55.617 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:55.618 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:55.618 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:55.618 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:55.618 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:55.618 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:55.618 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:55.618 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:55.618 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:55.618 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:55.618 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:55.618 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:55.618 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:55.618 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:55.618 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:55.619 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:55.619 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:55.619 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:55.619 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:55.619 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:55.619 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:55.619 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:55.619 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:55.619 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:55.619 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:55.620 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:55.620 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:55.620 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:55.620 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:55.620 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:55.620 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:55.620 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:55.620 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:55.620 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:55.620 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:55.620 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:55.620 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:55.620 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:55.620 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:55.621 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:55.621 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:55.621 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:55.621 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:55.621 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:55.621 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:55.621 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:55.621 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:55.621 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:55.621 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:55.621 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:55.621 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:55.621 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:55.621 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:55.622 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:55.622 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:55.622 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:55.622 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:55.622 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:55.622 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:55.622 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:55.622 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:55.622 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:55.622 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:55.623 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:55.623 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:55.623 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:55.623 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:55.623 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:55.623 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:55.623 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:55.623 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:55.623 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:55.623 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:55.623 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:55.623 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:55.623 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:55.624 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:55.624 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:55.624 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:55.624 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:55.624 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:55.624 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:55.624 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:55.624 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:55.624 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:55.624 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:55.624 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:55.624 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:55.625 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:55.625 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:55.625 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:55.625 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:55.625 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:55.625 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:55.625 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:55.625 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:55.625 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:55.625 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:55.625 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:55 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:55.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:55.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:55.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:55.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:55.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:55.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:55.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:55.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:55.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:55.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:55.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:55.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:55.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:55.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:55.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:55.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:55.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:55.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:55.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:55.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:55.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:55.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:55.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:55.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:55.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:55.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:55.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:55.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:55.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:55.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:55.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:55.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:55.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:55.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:55.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:55.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:55.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:55.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:55.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:55.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:55.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:55.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:55.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:55.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:55.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:55.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:55.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:55.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:55.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:55.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:55.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:55.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:55.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:55 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:56.405 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:56.405 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:56.405 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:56.406 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:56.406 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:56.406 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:56.406 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:56.406 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:56.406 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:56.406 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:56.406 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:56.406 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:56.406 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:56.407 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:56.407 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:56.407 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:56.407 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:56.407 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:56.407 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:56.407 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:56.407 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:56.407 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:56.407 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:56.407 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:56.407 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:56.407 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:56.408 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:56.408 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:56.408 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:56.408 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:56.408 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:56.408 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:56.408 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:56.408 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:56.408 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:56.408 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:56.408 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:56.408 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:56.408 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:56.408 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:56.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:56.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:56.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:56.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:56.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:56.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:56.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:56.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:56.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:56.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:56.695 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:56.695 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:56.695 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:56.696 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:56.696 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:56.696 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:56.696 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:56.696 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:56.696 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:56.696 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:56.696 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:56.696 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:56.696 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:56.696 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:56.696 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:56.696 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:56.697 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:56.697 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:56.697 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:56.697 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:56.697 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:56.697 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:56.697 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:56.697 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:56.697 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:56.697 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:56.697 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:56.697 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:56.697 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:56.698 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:56.698 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:56.698 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:56.698 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:56.698 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:56.698 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:56.698 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:56.698 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:56.698 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:56.698 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:56.699 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:56.699 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:56.699 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:56.699 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:56.699 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:56.699 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:56.699 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:56.699 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:56.699 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:56.699 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:56.699 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:56.699 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:56.699 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:56.699 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:56.700 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:56.700 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:56.700 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:56.700 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:56.700 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:56.700 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:56.700 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:56.700 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:56.700 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:56.700 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:56.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:56.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:56.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:56.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:56.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:56.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:56.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:56.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:56.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:56.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:56.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:56.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:56.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:56.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:56.702 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:56.702 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:56.702 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:56.702 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:56.702 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:56.702 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:56.702 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:56.702 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:56.702 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:56.702 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:56.702 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:56.703 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:56.703 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:56.703 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:56.703 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:56.703 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:56.703 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:56.703 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:56.703 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:56.703 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:56.703 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:56.703 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:56.703 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:56 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:57.085 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:04:56 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:04:56] "GET /metrics HTTP/1.1" 200 38262 "" "Prometheus/2.33.4" 2024-06-01T02:04:57.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:57.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:57.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:57.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:57.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:57.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:57.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:57.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:57.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:57.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:57.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:57.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:57.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:57.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:57.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:57.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:57.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:57.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:57.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:57.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:57.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:57.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:57.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:57.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:57.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:57.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:57.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:57.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:57.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:57.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:57.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:57.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:57.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:57.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:57.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:57.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:57.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:57.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:57.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:57.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:57.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:57.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:57.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:57.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:57.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:57.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:57.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:57.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:57.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:57.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:57.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:57.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:57.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:57.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:57.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:57.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:57.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:57.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:57.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:57.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:57.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:57.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:57.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:57.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:57.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:57.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:57.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:57.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:57.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:57.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:57.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:57.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:57.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:57.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:57.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:57.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:57.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:57.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:57.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:57.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:57.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:57.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:57.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:57.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:57.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:57.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:57.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:57.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:57.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:57.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:57.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:57.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:57.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:57.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:57.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:57.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:57 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:57.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:57.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:57.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:57.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:57.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:57.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:57.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:57.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:57.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:57.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:57.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:57.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:57.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:57.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:57.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:57.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:57.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:57.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:57.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:57.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:57.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:57.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:57.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:04:57.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:57.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:57.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:57.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:57.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:57.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:57.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:57.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:57.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:57.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:57.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:57.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:57.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:57.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:57.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:57.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:57.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:57.701 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:57.701 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:57.701 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:57.701 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:57.701 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:57.701 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:57.701 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:57.701 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:57 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.443 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:58.443 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:58.443 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:58.443 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:58.443 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:58.443 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:58.443 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:58.443 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:58.443 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.444 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:58.444 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:58.444 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:58.444 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:58.444 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:58.444 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.444 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:58.444 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:58.444 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:58.444 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:58.444 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:58.444 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.444 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:58.445 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:58.445 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:58.445 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:58.445 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:58.445 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.445 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:58.445 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:58.445 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:58.445 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:58.446 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:58.446 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:58.446 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:58.446 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:58.446 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:58.446 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:58.446 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:58.446 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.446 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:58.446 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:58.446 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:58.446 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:58.446 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:58.447 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.447 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:58.447 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:58.447 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:58.447 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:58.447 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:58.447 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.447 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:58.447 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:58.447 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:58.447 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:58.447 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:58.447 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.447 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:58.448 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:58.448 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:58.448 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:58.448 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:58.448 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.448 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.448 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.448 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.448 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:58.448 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:58.448 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:58.449 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:58.449 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.449 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:58.449 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:58.449 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:58.449 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:58.449 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.449 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:58.449 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:58.449 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:58.449 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:58.449 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.450 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:58.450 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:58.450 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.450 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.450 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.450 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.450 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:58.450 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:58.450 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:58.450 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:58.450 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.450 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:58.450 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:58.451 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:58.451 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:58.451 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.451 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:58.451 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:58.451 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:58.451 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:58.451 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:58 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:58.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:58.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:58.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:58.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:58.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:58.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:58.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:58.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:58.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:58.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:58.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:58.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:58.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:58.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:58.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:58.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:58.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:58.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:58.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:58.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:58.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:58.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:58.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:58.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:58.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:58.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:58.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:58.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:58.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:58.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:58.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:58.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:58.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:58.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:58.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:58.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:58.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:58.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:58.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:58.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:58.945 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:04:58 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:04:58.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"172.21.15.161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"smithi161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:04:59.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:59.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:59.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:59.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:59.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:59.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:59.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:59.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:59.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:59.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:59.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:59.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:59.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:59.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:59.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:59.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:59.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:59.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:59.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:59.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:59.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:59.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:59.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:59.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:59.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:59.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:59.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:59.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:59.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:04:59.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:59.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:59.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:59.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:59.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:59.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:59.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:59.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:59.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:59.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:59.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:59.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:59.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:59.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:59.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:59.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:59.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:59.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:59.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:59.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:59.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:59.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:59.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:59.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:59.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:59.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:59.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:59.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:59.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:59.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:04:59.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:59.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:59.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:59.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:59.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:59.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:59.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:59.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:59.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:59.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:59.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:59.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:59.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:59.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:59.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:59.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:59.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:59.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:59.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:59.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:59.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:59.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:59.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:59.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:59.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:59.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:59.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:59.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:59.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:59.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:59.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:59.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:59.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:59.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:59.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:59.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:59.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:59.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:59.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:59.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:59.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:59.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:59.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:59.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:59.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:59.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:59.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:59.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:59.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:59.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:59.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:04:59 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:59.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:59.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:59.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:59.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:59.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:59.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:59.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:59.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:59.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:59.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:59.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:59.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:59.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:59.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:59.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:59.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:59.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:59.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:59.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:59.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:59.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:59.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:59.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:59.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:59.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:59.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:59.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:59.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:59.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:04:59.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:59.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:59.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:59.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:59.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:59.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:04:59.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:59.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:59.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:59.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:59.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:59.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:59.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:59.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:59.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:59.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:59.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:59.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:59.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:59.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:59.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:04:59.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:04:59.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:04:59.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:04:59.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:04:59.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:04:59 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:00.627 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:00.628 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:00.628 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:00.628 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:00.628 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:00.628 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:00.628 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:00.628 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:00.628 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:00.628 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:00.628 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:00.628 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:00.629 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:00.629 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:00.629 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:00.629 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:00.629 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:00.629 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:00.629 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:00.629 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:00.629 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:00.629 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:00.629 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:00.629 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:00.629 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:00.630 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:00.630 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:00.630 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:00.630 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:00.630 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:00.630 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:00.630 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:00.630 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:00.630 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:00.631 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:00.631 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:00.631 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:00.631 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:00.631 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:00.631 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:00.631 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:00.631 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:00.631 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:00.631 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:00.631 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:00.631 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:00.631 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:00.631 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:00.632 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:00.632 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:00.632 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:00.632 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:00.632 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:00.632 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:00.632 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:00.632 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:00.632 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:00.632 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:00.632 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:00.632 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:00.633 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:00.633 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:00.633 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:00.633 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:00.633 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:00.633 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:00.633 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:00.633 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:00.633 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:00.633 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:00.633 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:00.634 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:00.634 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:00.634 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:00.634 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:00.634 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:00.634 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:00.634 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:00.634 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:00.634 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:00.634 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:00.634 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:00.634 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:00.634 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:00.635 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:00.635 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:00.635 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:00.635 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:00.635 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:00.635 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:00.635 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:00.635 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:00.635 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:00.635 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:00.635 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:00.636 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:00.636 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:00.636 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:00.636 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:00.636 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:00.636 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:00.636 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:00.636 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:00.636 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:00.636 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:00.636 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:00.636 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:00.636 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:00 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:00.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:00.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:00.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:00.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:00.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:00.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:00.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:00.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:00.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:00.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:00.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:00.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:00.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:00.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:00.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:00.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:00.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:00.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:00.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:00.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:00.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:00.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:00.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:00.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:00.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:00.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:00.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:00.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:00.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:00.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:00.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:00.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:00.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:00.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:00.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:00.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:00.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:00.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:00.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:00.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:00.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:00.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:00.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:00.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:00.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:00.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:00.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:00.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:00.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:00.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:00.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:00.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:00.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:00.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:00 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:01.682 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:01.682 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:01.682 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:01.682 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:01.683 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:01.683 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:01.683 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:01.683 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:01.683 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:01.683 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:01.683 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:01.683 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:01.683 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:01.683 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:01.683 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:01.683 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:01.683 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:01.684 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:01.684 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:01.684 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:01.684 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:01.684 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:01.684 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:01.684 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:01.684 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:01.684 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:01.684 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:01.684 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:01.684 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:01.684 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:01.685 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:01.685 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:01.685 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:01.685 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:01.685 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:01.685 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:01.685 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:01.685 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:01.685 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:01.685 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:01.686 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:01.686 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:01.686 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:01.686 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:01.686 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:01.686 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:01.686 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:01.686 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:01.686 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:01.686 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:01.686 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:01.686 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:01.686 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:01.687 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:01.687 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:01.687 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:01.687 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:01.687 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:01.687 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:01.687 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:01.688 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:01.688 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:01.688 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:01.688 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:01.688 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:01.688 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:05:01.688 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:01.688 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:01.688 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:01.688 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:01.688 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:01.688 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:01.688 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:01.689 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:01.689 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:01.689 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:01.689 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:01.689 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:01.689 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:01.689 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:01.689 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:01.689 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:01.689 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:01.689 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:01.689 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:01.689 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:01.689 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:01.690 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:01.690 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:01.690 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:01.690 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:01.690 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:01.690 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:01.690 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:01.690 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:01.691 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:01.691 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:05:01.691 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:01.691 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:01.691 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:01.691 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:01.691 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:01.691 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:01.691 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:01.691 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:01.691 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:01.691 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:01.692 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:01.692 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:01.692 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:01.692 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:01.692 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:01.692 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:01.692 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:01.692 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:01.692 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:01.692 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:01.692 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:01.692 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:01.692 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:01.692 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:01.693 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:01 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:01.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:01.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:01.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:01.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:01.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:01.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:01.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:01.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:01.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:01.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:01.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:01.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:01.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:01.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:01.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:01.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:01.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:01.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:01.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:01.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:01.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:01.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:01.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:01.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:01.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:01.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:01.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:01.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:01.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:01.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:01.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:01.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:01.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:01.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:01.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:01.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:05:01.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:01.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:01.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:01.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:01.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:01.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:01.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:01.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:01.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:01.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:01.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:01.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:01.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:01.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:01.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:01.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:01.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:01.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:01.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:01.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:01.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:01.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:01.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:01.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:01.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:01 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:02.648 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:02.649 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:02.649 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:02.649 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:02.649 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:02.649 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:02.649 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:02.649 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:02.649 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:02.649 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:02.649 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:02.649 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:02.649 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:02.650 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:02.650 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:02.650 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:02.650 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:02.650 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:02.650 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:02.650 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:02.650 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:02.650 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:02.650 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:02.650 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:02.650 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:02.650 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:02.651 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:02.651 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:02.651 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:02.651 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:05:02.651 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:02.651 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:02.651 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:02.651 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:02.651 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:02.651 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:02.652 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:02.652 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:02.652 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:02.652 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:02.652 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:02.652 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:02.652 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:02.652 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:02.652 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:02.652 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:02.652 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:02.652 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:02.652 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:02.653 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:02.653 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:02.653 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:02.653 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:02.653 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:02.653 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:02.653 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:02.653 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:02.653 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:02.653 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:02.653 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:05:02.654 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:02.654 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:02.654 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:02.654 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:02.654 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:02.654 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:02.654 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:02.654 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:02.655 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:02.655 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:02.655 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:02.655 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:02.655 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:02.655 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:02.655 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:02.655 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:02.655 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:02.655 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:02.655 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:02.655 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:02.655 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:02.655 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:02.656 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:02.656 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:02.656 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:02.656 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:02.656 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:02.656 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:02.656 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:02.656 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:02.656 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:02.656 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:02.656 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:02.656 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:02.657 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:02.657 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:02.657 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:02.657 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:02.657 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:02.657 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:02.657 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:02.657 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:02.657 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:02.657 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:02.657 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:02.657 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:02.657 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:02.658 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:02.658 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:02.658 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:02.658 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:02.658 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:02 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:02.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:02.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:02.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:02.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:02.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:02.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:02.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:02.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:02.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:02.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:02.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:02.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:02.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:02.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:02.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:02.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:02.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:02.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:02.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:02.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:02.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:02.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:02.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:02.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:02.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:02.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:02.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:02.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:02.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:02.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:05:02.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:02.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:02.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:02.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:02.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:02.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:02.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:02.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:02.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:02.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:02.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:02.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:02.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:02.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:02.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:02.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:02.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:02.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:02.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:02.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:02.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:02.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:02.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:02.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:02.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:02.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:02 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:03.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:03.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:03.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:03.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:03.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:03.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:03.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:03.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:03.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:03.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:03.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:03.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:03.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:03.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:03.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:03.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:03.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:03.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:03.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:03.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:03.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:03.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:03.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:03.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:03.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:03.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:03.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:03.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:03.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:03.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:03.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:03.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:03.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:03.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:03.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:03.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:03.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:03.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:03.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:03.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:03.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:03.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:03.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:03.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:03.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:03.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:03.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:03.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:03.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:03.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:03.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:03.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:03.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:03.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:03.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:03.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:03.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:03.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:03.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:03.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:03.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:03.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:03.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:03.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:03.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:03.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:03.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:03.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:03.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:03.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:03.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:03.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:03.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:03.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:03.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:03.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:03.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:03.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:03.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:03.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:03.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:03.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:03.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:03.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:03.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:03.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:03.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:03.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:03.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:03.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:03.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:03.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:03.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:03.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:03.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:03.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:03.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:03.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:03.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:03.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:03.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:03.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:03.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:03.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:03.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:03.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:03.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:03.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:03.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:03.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:03.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:03.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:03.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:03.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:03.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:03.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:03 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:03.694 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:05:03.505Z caller=manager.go:609 level=warn component="rule manager" group=nodes msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on(instance) group_left(nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi116\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:05:03.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:03.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:03.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:03.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:03.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:03.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:03.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:03.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:03.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:03.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:03.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:03.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:03.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:03.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:03.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:03.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:03.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:03.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:03.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:03.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:03.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:03.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:03.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:03.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:03.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:03.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:03.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:03.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:03.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:03.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:03.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:03.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:03.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:03.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:03.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:03.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:03.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:03.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:03.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:03.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:03.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:03.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:03.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:03.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:03.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:03.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:03.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:03.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:03.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:03.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:03.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:03.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:03.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:03.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:03.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:03.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:03.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:03.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:03 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:04.684 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:04.684 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:04.684 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:04.684 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:04.684 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:04.684 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:04.684 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:04.684 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:04.684 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:04.684 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:04.685 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:04.685 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:04.685 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:04.685 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:04.685 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:04.685 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:04.685 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:04.685 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:04.685 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:04.685 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:04.685 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:04.685 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:04.685 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:04.686 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:04.686 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:04.686 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:04.686 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:04.686 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:04.686 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:04.686 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:04.686 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:04.686 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:04.686 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:04.686 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:04.686 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:04.687 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:04.687 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-06-01T02:05:04.687 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon ok-to-stop", "ids": ["c"]}]: dispatch 2024-06-01T02:05:04.687 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:04.687 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:04.687 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:04.687 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:04.687 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:04.687 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:04.688 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:04.688 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:04.688 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:04.688 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:04.688 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:04.688 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:04.688 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:04.688 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:04.688 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:04.688 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:04.688 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:04.688 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:04.688 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:04.689 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:04.689 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:04.689 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:04.689 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:04.689 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:04.689 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:04.689 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:04.689 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:04.689 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:04.689 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:04.689 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:04.689 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:04.689 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:04.690 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:04.690 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:04.690 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:04.690 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:04.690 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-06-01T02:05:04.690 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon ok-to-stop", "ids": ["c"]}]: dispatch 2024-06-01T02:05:04.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:04.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:04.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:04.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:04.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:04.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:04.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:04.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:04.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:04.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:04.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:04.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:04.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:04.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:04.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:04.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:04.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:04.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:04.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:04.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:04.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:04.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:04.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:04.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-06-01T02:05:04.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:04 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon ok-to-stop", "ids": ["c"]}]: dispatch 2024-06-01T02:05:05.660 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:05 smithi116 ceph-mon[29958]: Upgrade: It appears safe to stop mon.c 2024-06-01T02:05:05.660 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:05 smithi116 ceph-mon[29958]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:05.660 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:05 smithi116 ceph-mon[89502]: Upgrade: It appears safe to stop mon.c 2024-06-01T02:05:05.660 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:05 smithi116 ceph-mon[89502]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:05.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:05 smithi161 ceph-mon[26055]: Upgrade: It appears safe to stop mon.c 2024-06-01T02:05:05.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:05 smithi161 ceph-mon[26055]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:07.060 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:05:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:05:06] "GET /metrics HTTP/1.1" 200 38253 "" "Prometheus/2.33.4" 2024-06-01T02:05:07.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:07 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:07.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:07 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:07.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:07 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:07.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:07 smithi161 ceph-mon[26055]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:07.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:07 smithi161 ceph-mon[26055]: Upgrade: Updating mon.c 2024-06-01T02:05:07.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:07 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:07.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:07 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T02:05:07.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:07 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T02:05:07.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:07 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:07.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:07 smithi161 ceph-mon[26055]: Deploying daemon mon.c on smithi116 2024-06-01T02:05:07.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:07 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:07.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:07 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:07.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:07 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:07.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:07 smithi116 ceph-mon[29958]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:07.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:07 smithi116 ceph-mon[29958]: Upgrade: Updating mon.c 2024-06-01T02:05:07.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:07 smithi116 ceph-mon[29958]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:07.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:07 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T02:05:07.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:07 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T02:05:07.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:07 smithi116 ceph-mon[29958]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:07.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:07 smithi116 ceph-mon[29958]: Deploying daemon mon.c on smithi116 2024-06-01T02:05:07.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:07 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:07.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:07 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:07.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:07 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:07.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:07 smithi116 ceph-mon[89502]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:07.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:07 smithi116 ceph-mon[89502]: Upgrade: Updating mon.c 2024-06-01T02:05:07.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:07 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:07.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T02:05:07.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T02:05:07.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:07.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:07 smithi116 ceph-mon[89502]: Deploying daemon mon.c on smithi116 2024-06-01T02:05:08.944 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:05:08 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:05:08.660Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"172.21.15.161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"smithi161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:05:10.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:09 smithi116 ceph-mon[89502]: pgmap v22: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:10.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:09 smithi116 ceph-mon[29958]: pgmap v22: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:10.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:09 smithi161 ceph-mon[26055]: pgmap v22: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:11.288 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:10 smithi116 systemd[1]: Stopping Ceph mon.c for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:05:12.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-c[29954]: 2024-06-01T02:05:11.628+0000 7fba0f098700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-01T02:05:12.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-c[29954]: 2024-06-01T02:05:11.628+0000 7fba0f098700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-06-01T02:05:12.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:11 smithi116 podman[95420]: 2024-06-01 02:05:11.843281567 +0000 UTC m=+0.330312844 container died 4a6fe0b16d63aae1feb9a083bd969990090f8f1ccf0a30d99bb56ceb5aefede1 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-c, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, vendor=Red Hat, Inc., io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., ceph=True, build-date=2022-05-03T08:36:31.336870, version=8, vcs-type=git, com.redhat.component=centos-stream-container, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, distribution-scope=public, architecture=x86_64, release=754, name=centos-stream, GIT_CLEAN=True, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, io.buildah.version=1.19.8, CEPH_POINT_RELEASE=-17.2.0, io.openshift.tags=base centos centos-stream, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, io.k8s.display-name=CentOS Stream 8, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.openshift.expose-services=) 2024-06-01T02:05:12.676 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:12 smithi116 podman[95420]: 2024-06-01 02:05:12.395853529 +0000 UTC m=+0.882884808 container cleanup 4a6fe0b16d63aae1feb9a083bd969990090f8f1ccf0a30d99bb56ceb5aefede1 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-c, io.openshift.expose-services=, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, io.openshift.tags=base centos centos-stream, release=754, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.component=centos-stream-container, maintainer=Guillaume Abrioux , version=8, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, io.buildah.version=1.19.8, CEPH_POINT_RELEASE=-17.2.0, name=centos-stream, build-date=2022-05-03T08:36:31.336870, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, distribution-scope=public, vcs-type=git, GIT_CLEAN=True, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, ceph=True, architecture=x86_64, RELEASE=HEAD, vendor=Red Hat, Inc., GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, io.k8s.display-name=CentOS Stream 8, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image) 2024-06-01T02:05:12.676 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:12 smithi116 bash[95420]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-c 2024-06-01T02:05:12.936 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:12 smithi116 podman[95432]: 2024-06-01 02:05:12.680716784 +0000 UTC m=+0.835541296 container remove 4a6fe0b16d63aae1feb9a083bd969990090f8f1ccf0a30d99bb56ceb5aefede1 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-c, io.k8s.display-name=CentOS Stream 8, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, GIT_CLEAN=True, version=8, distribution-scope=public, RELEASE=HEAD, build-date=2022-05-03T08:36:31.336870, ceph=True, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, architecture=x86_64, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, io.openshift.tags=base centos centos-stream, io.buildah.version=1.19.8, name=centos-stream, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, com.redhat.component=centos-stream-container, vcs-type=git, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.license_terms=https://centos.org/legal/licensing-policy/, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, vendor=Red Hat, Inc., release=754, CEPH_POINT_RELEASE=-17.2.0, io.openshift.expose-services=) 2024-06-01T02:05:12.936 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:12 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mon.c.service: Deactivated successfully. 2024-06-01T02:05:12.936 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:12 smithi116 systemd[1]: Stopped Ceph mon.c for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:05:12.936 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:12 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mon.c.service: Consumed 10.592s CPU time. 2024-06-01T02:05:13.251 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 systemd[1]: Starting Ceph mon.c for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:05:13.505 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 podman[95562]: 2024-06-01 02:05:13.158451532 +0000 UTC m=+0.016934655 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:05:13.505 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 podman[95562]: 2024-06-01 02:05:13.265206512 +0000 UTC m=+0.123689644 container create 8c572b16c86d44474cee4a9d6f0a031701d5c8fadb231e77c06ce7450e88da08 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc) 2024-06-01T02:05:13.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 podman[95562]: 2024-06-01 02:05:13.509692413 +0000 UTC m=+0.368175536 container init 8c572b16c86d44474cee4a9d6f0a031701d5c8fadb231e77c06ce7450e88da08 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-01T02:05:13.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 podman[95562]: 2024-06-01 02:05:13.513404007 +0000 UTC m=+0.371887118 container start 8c572b16c86d44474cee4a9d6f0a031701d5c8fadb231e77c06ce7450e88da08 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-c, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-06-01T02:05:13.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: set uid:gid to 167:167 (ceph:ceph) 2024-06-01T02:05:13.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc), process ceph-mon, pid 2 2024-06-01T02:05:13.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: pidfile_write: ignore empty --pid-file 2024-06-01T02:05:13.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: load: jerasure load: lrc 2024-06-01T02:05:13.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: RocksDB version: 7.9.2 2024-06-01T02:05:13.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Git sha 0 2024-06-01T02:05:13.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Compile date 2024-05-30 15:42:47 2024-06-01T02:05:13.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: DB SUMMARY 2024-06-01T02:05:13.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: DB Session ID: 4JTLRKXU1IVWYR4FD83R 2024-06-01T02:05:13.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: CURRENT file: CURRENT 2024-06-01T02:05:13.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: IDENTITY file: IDENTITY 2024-06-01T02:05:13.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: MANIFEST file: MANIFEST-000009 size: 2048 Bytes 2024-06-01T02:05:13.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 1, files: 000042.sst 2024-06-01T02:05:13.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000040.log size: 646918 ; 2024-06-01T02:05:13.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.error_if_exists: 0 2024-06-01T02:05:13.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.create_if_missing: 0 2024-06-01T02:05:13.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.paranoid_checks: 1 2024-06-01T02:05:13.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-01T02:05:13.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-01T02:05:13.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-01T02:05:13.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.env: 0x55966a8e5c60 2024-06-01T02:05:13.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.fs: PosixFileSystem 2024-06-01T02:05:13.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.info_log: 0x55966c125b20 2024-06-01T02:05:13.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.max_file_opening_threads: 16 2024-06-01T02:05:13.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.statistics: (nil) 2024-06-01T02:05:13.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.use_fsync: 0 2024-06-01T02:05:13.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.max_log_file_size: 0 2024-06-01T02:05:13.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-01T02:05:13.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-01T02:05:13.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.keep_log_file_num: 1000 2024-06-01T02:05:13.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.recycle_log_file_num: 0 2024-06-01T02:05:13.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.allow_fallocate: 1 2024-06-01T02:05:13.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.allow_mmap_reads: 0 2024-06-01T02:05:13.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.allow_mmap_writes: 0 2024-06-01T02:05:13.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.use_direct_reads: 0 2024-06-01T02:05:13.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-01T02:05:13.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.create_missing_column_families: 0 2024-06-01T02:05:13.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.db_log_dir: 2024-06-01T02:05:13.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.wal_dir: 2024-06-01T02:05:13.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-01T02:05:13.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-01T02:05:13.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-01T02:05:13.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-01T02:05:13.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-01T02:05:13.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-01T02:05:13.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.advise_random_on_open: 1 2024-06-01T02:05:13.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.db_write_buffer_size: 0 2024-06-01T02:05:13.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.write_buffer_manager: 0x55966c0f6820 2024-06-01T02:05:13.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-01T02:05:13.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-01T02:05:13.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-01T02:05:13.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.rate_limiter: (nil) 2024-06-01T02:05:13.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-01T02:05:13.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.wal_recovery_mode: 2 2024-06-01T02:05:13.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.enable_thread_tracking: 0 2024-06-01T02:05:13.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.enable_pipelined_write: 0 2024-06-01T02:05:13.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.unordered_write: 0 2024-06-01T02:05:13.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-01T02:05:13.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-01T02:05:13.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-01T02:05:13.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-01T02:05:13.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.row_cache: None 2024-06-01T02:05:13.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.wal_filter: None 2024-06-01T02:05:13.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-01T02:05:13.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.allow_ingest_behind: 0 2024-06-01T02:05:13.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.two_write_queues: 0 2024-06-01T02:05:13.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.manual_wal_flush: 0 2024-06-01T02:05:13.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.wal_compression: 0 2024-06-01T02:05:13.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.atomic_flush: 0 2024-06-01T02:05:13.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-01T02:05:13.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-01T02:05:13.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-01T02:05:13.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.log_readahead_size: 0 2024-06-01T02:05:13.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-01T02:05:13.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.best_efforts_recovery: 0 2024-06-01T02:05:13.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-01T02:05:13.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-01T02:05:13.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.allow_data_in_errors: 0 2024-06-01T02:05:13.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.db_host_id: __hostname__ 2024-06-01T02:05:13.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-01T02:05:13.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.max_background_jobs: 2 2024-06-01T02:05:13.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.max_background_compactions: -1 2024-06-01T02:05:13.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.max_subcompactions: 1 2024-06-01T02:05:13.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-01T02:05:13.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-01T02:05:13.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-01T02:05:13.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.max_total_wal_size: 0 2024-06-01T02:05:13.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-01T02:05:13.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-01T02:05:13.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-01T02:05:13.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-01T02:05:13.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.max_open_files: -1 2024-06-01T02:05:13.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.bytes_per_sync: 0 2024-06-01T02:05:13.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-01T02:05:13.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-01T02:05:13.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.compaction_readahead_size: 0 2024-06-01T02:05:13.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.max_background_flushes: -1 2024-06-01T02:05:13.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Compression algorithms supported: 2024-06-01T02:05:13.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-01T02:05:13.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: kZSTD supported: 0 2024-06-01T02:05:13.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: kXpressCompression supported: 0 2024-06-01T02:05:13.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: kLZ4HCCompression supported: 1 2024-06-01T02:05:13.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: kZlibCompression supported: 1 2024-06-01T02:05:13.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: kSnappyCompression supported: 1 2024-06-01T02:05:13.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: kLZ4Compression supported: 1 2024-06-01T02:05:13.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: kBZip2Compression supported: 0 2024-06-01T02:05:13.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-01T02:05:13.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-01T02:05:13.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000009 2024-06-01T02:05:13.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-01T02:05:13.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-01T02:05:13.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.merge_operator: 2024-06-01T02:05:13.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.compaction_filter: None 2024-06-01T02:05:13.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.compaction_filter_factory: None 2024-06-01T02:05:13.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.sst_partitioner_factory: None 2024-06-01T02:05:13.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-01T02:05:13.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-01T02:05:13.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55966c125c40) 2024-06-01T02:05:13.803 INFO:journalctl@ceph.mon.c.smithi116.stdout: cache_index_and_filter_blocks: 1 2024-06-01T02:05:13.803 INFO:journalctl@ceph.mon.c.smithi116.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-01T02:05:13.804 INFO:journalctl@ceph.mon.c.smithi116.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-01T02:05:13.804 INFO:journalctl@ceph.mon.c.smithi116.stdout: pin_top_level_index_and_filter: 1 2024-06-01T02:05:13.804 INFO:journalctl@ceph.mon.c.smithi116.stdout: index_type: 0 2024-06-01T02:05:13.804 INFO:journalctl@ceph.mon.c.smithi116.stdout: data_block_index_type: 0 2024-06-01T02:05:13.804 INFO:journalctl@ceph.mon.c.smithi116.stdout: index_shortening: 1 2024-06-01T02:05:13.804 INFO:journalctl@ceph.mon.c.smithi116.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-01T02:05:13.804 INFO:journalctl@ceph.mon.c.smithi116.stdout: checksum: 4 2024-06-01T02:05:13.804 INFO:journalctl@ceph.mon.c.smithi116.stdout: no_block_cache: 0 2024-06-01T02:05:13.804 INFO:journalctl@ceph.mon.c.smithi116.stdout: block_cache: 0x55966c127090 2024-06-01T02:05:13.804 INFO:journalctl@ceph.mon.c.smithi116.stdout: block_cache_name: BinnedLRUCache 2024-06-01T02:05:13.804 INFO:journalctl@ceph.mon.c.smithi116.stdout: block_cache_options: 2024-06-01T02:05:13.804 INFO:journalctl@ceph.mon.c.smithi116.stdout: capacity : 536870912 2024-06-01T02:05:13.804 INFO:journalctl@ceph.mon.c.smithi116.stdout: num_shard_bits : 4 2024-06-01T02:05:13.804 INFO:journalctl@ceph.mon.c.smithi116.stdout: strict_capacity_limit : 0 2024-06-01T02:05:13.805 INFO:journalctl@ceph.mon.c.smithi116.stdout: high_pri_pool_ratio: 0.000 2024-06-01T02:05:13.805 INFO:journalctl@ceph.mon.c.smithi116.stdout: block_cache_compressed: (nil) 2024-06-01T02:05:13.805 INFO:journalctl@ceph.mon.c.smithi116.stdout: persistent_cache: (nil) 2024-06-01T02:05:13.805 INFO:journalctl@ceph.mon.c.smithi116.stdout: block_size: 4096 2024-06-01T02:05:13.805 INFO:journalctl@ceph.mon.c.smithi116.stdout: block_size_deviation: 10 2024-06-01T02:05:13.805 INFO:journalctl@ceph.mon.c.smithi116.stdout: block_restart_interval: 16 2024-06-01T02:05:13.805 INFO:journalctl@ceph.mon.c.smithi116.stdout: index_block_restart_interval: 1 2024-06-01T02:05:13.805 INFO:journalctl@ceph.mon.c.smithi116.stdout: metadata_block_size: 4096 2024-06-01T02:05:13.805 INFO:journalctl@ceph.mon.c.smithi116.stdout: partition_filters: 0 2024-06-01T02:05:13.805 INFO:journalctl@ceph.mon.c.smithi116.stdout: use_delta_encoding: 1 2024-06-01T02:05:13.805 INFO:journalctl@ceph.mon.c.smithi116.stdout: filter_policy: bloomfilter 2024-06-01T02:05:13.805 INFO:journalctl@ceph.mon.c.smithi116.stdout: whole_key_filtering: 1 2024-06-01T02:05:13.805 INFO:journalctl@ceph.mon.c.smithi116.stdout: verify_compression: 0 2024-06-01T02:05:13.805 INFO:journalctl@ceph.mon.c.smithi116.stdout: read_amp_bytes_per_bit: 0 2024-06-01T02:05:13.805 INFO:journalctl@ceph.mon.c.smithi116.stdout: format_version: 5 2024-06-01T02:05:13.806 INFO:journalctl@ceph.mon.c.smithi116.stdout: enable_index_compression: 1 2024-06-01T02:05:13.806 INFO:journalctl@ceph.mon.c.smithi116.stdout: block_align: 0 2024-06-01T02:05:13.806 INFO:journalctl@ceph.mon.c.smithi116.stdout: max_auto_readahead_size: 262144 2024-06-01T02:05:13.806 INFO:journalctl@ceph.mon.c.smithi116.stdout: prepopulate_block_cache: 0 2024-06-01T02:05:13.806 INFO:journalctl@ceph.mon.c.smithi116.stdout: initial_auto_readahead_size: 8192 2024-06-01T02:05:13.806 INFO:journalctl@ceph.mon.c.smithi116.stdout: num_file_reads_for_auto_readahead: 2 2024-06-01T02:05:13.806 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.write_buffer_size: 33554432 2024-06-01T02:05:13.806 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.max_write_buffer_number: 2 2024-06-01T02:05:13.806 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.compression: NoCompression 2024-06-01T02:05:13.806 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.bottommost_compression: Disabled 2024-06-01T02:05:13.806 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.prefix_extractor: nullptr 2024-06-01T02:05:13.806 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-01T02:05:13.806 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.num_levels: 7 2024-06-01T02:05:13.806 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-01T02:05:13.807 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-01T02:05:13.807 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-01T02:05:13.807 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-01T02:05:13.807 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-01T02:05:13.807 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-01T02:05:13.807 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-01T02:05:13.807 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-01T02:05:13.807 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-01T02:05:13.807 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-01T02:05:13.807 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-01T02:05:13.807 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-01T02:05:13.807 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-01T02:05:13.807 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.compression_opts.level: 32767 2024-06-01T02:05:13.807 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.compression_opts.strategy: 0 2024-06-01T02:05:13.808 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-01T02:05:13.808 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-01T02:05:13.808 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-01T02:05:13.808 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-01T02:05:13.808 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.compression_opts.enabled: false 2024-06-01T02:05:13.808 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-01T02:05:13.808 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-01T02:05:13.808 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-01T02:05:13.808 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-01T02:05:13.808 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.target_file_size_base: 67108864 2024-06-01T02:05:13.808 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-01T02:05:13.808 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-01T02:05:13.808 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-01T02:05:13.808 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-01T02:05:13.809 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-01T02:05:13.809 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-01T02:05:13.809 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-01T02:05:13.809 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-01T02:05:13.809 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-01T02:05:13.809 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-01T02:05:13.809 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-01T02:05:13.809 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-01T02:05:13.809 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-01T02:05:13.809 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-01T02:05:13.809 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.arena_block_size: 1048576 2024-06-01T02:05:13.809 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-01T02:05:13.809 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-01T02:05:13.809 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.disable_auto_compactions: 0 2024-06-01T02:05:13.809 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-01T02:05:13.809 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-01T02:05:13.810 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-01T02:05:13.810 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-01T02:05:13.810 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-01T02:05:13.810 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-01T02:05:13.810 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-01T02:05:13.810 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-01T02:05:13.810 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-01T02:05:13.810 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-01T02:05:13.810 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.table_properties_collectors: 2024-06-01T02:05:13.810 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.inplace_update_support: 0 2024-06-01T02:05:13.810 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-01T02:05:13.810 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-01T02:05:13.810 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-01T02:05:13.810 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-01T02:05:13.811 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.bloom_locality: 0 2024-06-01T02:05:13.811 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.max_successive_merges: 0 2024-06-01T02:05:13.811 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-01T02:05:13.811 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.paranoid_file_checks: 0 2024-06-01T02:05:13.811 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.force_consistency_checks: 1 2024-06-01T02:05:13.811 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.report_bg_io_stats: 0 2024-06-01T02:05:13.811 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.ttl: 2592000 2024-06-01T02:05:13.811 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-01T02:05:13.811 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-01T02:05:13.811 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-01T02:05:13.811 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.enable_blob_files: false 2024-06-01T02:05:13.811 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.min_blob_size: 0 2024-06-01T02:05:13.811 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.blob_file_size: 268435456 2024-06-01T02:05:13.811 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-01T02:05:13.811 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-01T02:05:13.812 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-01T02:05:13.812 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-01T02:05:13.812 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-01T02:05:13.812 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.blob_file_starting_level: 0 2024-06-01T02:05:13.812 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-01T02:05:13.812 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 42.sst 2024-06-01T02:05:13.812 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-06-01T02:05:13.812 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 44, last_sequence is 25925, log_number is 40,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-06-01T02:05:13.812 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 40 2024-06-01T02:05:13.812 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b0f68023-9ac6-4086-b6ba-2926fe5078e1 2024-06-01T02:05:13.812 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717207513543564, "job": 1, "event": "recovery_started", "wal_files": [40]} 2024-06-01T02:05:13.812 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #40 mode 2 2024-06-01T02:05:13.812 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717207513545930, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 45, "file_size": 409047, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 25930, "largest_seqno": 26347, "table_properties": {"data_size": 406497, "index_size": 896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 6238, "raw_average_key_size": 25, "raw_value_size": 401966, "raw_average_value_size": 1627, "num_data_blocks": 38, "num_entries": 247, "num_filter_entries": 247, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1717207513, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b0f68023-9ac6-4086-b6ba-2926fe5078e1", "db_session_id": "4JTLRKXU1IVWYR4FD83R", "orig_file_number": 45, "seqno_to_time_mapping": "N/A"}} 2024-06-01T02:05:13.812 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717207513546016, "job": 1, "event": "recovery_finished"} 2024-06-01T02:05:13.812 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: [db/version_set.cc:5047] Creating manifest 47 2024-06-01T02:05:13.813 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-06-01T02:05:13.813 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000040.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-01T02:05:13.813 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55966c216000 2024-06-01T02:05:13.813 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: DB pointer 0x55966c200000 2024-06-01T02:05:13.813 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-01T02:05:13.813 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-01T02:05:13.813 INFO:journalctl@ceph.mon.c.smithi116.stdout: ** DB Stats ** 2024-06-01T02:05:13.813 INFO:journalctl@ceph.mon.c.smithi116.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-01T02:05:13.813 INFO:journalctl@ceph.mon.c.smithi116.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-01T02:05:13.813 INFO:journalctl@ceph.mon.c.smithi116.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-01T02:05:13.813 INFO:journalctl@ceph.mon.c.smithi116.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-01T02:05:13.813 INFO:journalctl@ceph.mon.c.smithi116.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-01T02:05:13.813 INFO:journalctl@ceph.mon.c.smithi116.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-01T02:05:13.813 INFO:journalctl@ceph.mon.c.smithi116.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-01T02:05:13.813 INFO:journalctl@ceph.mon.c.smithi116.stdout: 2024-06-01T02:05:13.814 INFO:journalctl@ceph.mon.c.smithi116.stdout: ** Compaction Stats [default] ** 2024-06-01T02:05:13.814 INFO:journalctl@ceph.mon.c.smithi116.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-01T02:05:13.814 INFO:journalctl@ceph.mon.c.smithi116.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-01T02:05:13.814 INFO:journalctl@ceph.mon.c.smithi116.stdout: L0 1/0 399.46 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 288.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-01T02:05:13.814 INFO:journalctl@ceph.mon.c.smithi116.stdout: L6 1/0 10.84 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-06-01T02:05:13.814 INFO:journalctl@ceph.mon.c.smithi116.stdout: Sum 2/0 11.23 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 288.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-01T02:05:13.814 INFO:journalctl@ceph.mon.c.smithi116.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 288.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-01T02:05:13.814 INFO:journalctl@ceph.mon.c.smithi116.stdout: 2024-06-01T02:05:13.814 INFO:journalctl@ceph.mon.c.smithi116.stdout: ** Compaction Stats [default] ** 2024-06-01T02:05:13.814 INFO:journalctl@ceph.mon.c.smithi116.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-01T02:05:13.814 INFO:journalctl@ceph.mon.c.smithi116.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-01T02:05:13.814 INFO:journalctl@ceph.mon.c.smithi116.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 288.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-01T02:05:13.814 INFO:journalctl@ceph.mon.c.smithi116.stdout: 2024-06-01T02:05:13.814 INFO:journalctl@ceph.mon.c.smithi116.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-01T02:05:13.815 INFO:journalctl@ceph.mon.c.smithi116.stdout: 2024-06-01T02:05:13.815 INFO:journalctl@ceph.mon.c.smithi116.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-01T02:05:13.815 INFO:journalctl@ceph.mon.c.smithi116.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-01T02:05:13.815 INFO:journalctl@ceph.mon.c.smithi116.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-01T02:05:13.815 INFO:journalctl@ceph.mon.c.smithi116.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-01T02:05:13.815 INFO:journalctl@ceph.mon.c.smithi116.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-01T02:05:13.815 INFO:journalctl@ceph.mon.c.smithi116.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-01T02:05:13.815 INFO:journalctl@ceph.mon.c.smithi116.stdout: Cumulative compaction: 0.00 GB write, 86.21 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-01T02:05:13.815 INFO:journalctl@ceph.mon.c.smithi116.stdout: Interval compaction: 0.00 GB write, 86.21 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-01T02:05:13.815 INFO:journalctl@ceph.mon.c.smithi116.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-01T02:05:13.815 INFO:journalctl@ceph.mon.c.smithi116.stdout: Block cache BinnedLRUCache@0x55966c127090#2 capacity: 512.00 MB usage: 1.67 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-06-01T02:05:13.815 INFO:journalctl@ceph.mon.c.smithi116.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.72 KB,0.000137091%) IndexBlock(1,0.95 KB,0.000181794%) Misc(1,0.00 KB,0%) 2024-06-01T02:05:13.815 INFO:journalctl@ceph.mon.c.smithi116.stdout: 2024-06-01T02:05:13.815 INFO:journalctl@ceph.mon.c.smithi116.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-01T02:05:13.816 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: starting mon.c rank 1 at public addrs [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] at bind addrs [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T02:05:13.816 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: mon.c@-1(???) e3 preinit fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T02:05:13.816 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: mon.c@-1(???).mds e1 new map 2024-06-01T02:05:13.816 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: mon.c@-1(???).mds e1 print_map 2024-06-01T02:05:13.816 INFO:journalctl@ceph.mon.c.smithi116.stdout: e1 2024-06-01T02:05:13.816 INFO:journalctl@ceph.mon.c.smithi116.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-01T02:05:13.816 INFO:journalctl@ceph.mon.c.smithi116.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-01T02:05:13.816 INFO:journalctl@ceph.mon.c.smithi116.stdout: legacy client fscid: -1 2024-06-01T02:05:13.816 INFO:journalctl@ceph.mon.c.smithi116.stdout: 2024-06-01T02:05:13.816 INFO:journalctl@ceph.mon.c.smithi116.stdout: No filesystems configured 2024-06-01T02:05:13.816 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: mon.c@-1(???).osd e114 crush map has features 3314933000854323200, adjusting msgr requires 2024-06-01T02:05:13.816 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: mon.c@-1(???).osd e114 crush map has features 432629239337189376, adjusting msgr requires 2024-06-01T02:05:13.816 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: mon.c@-1(???).osd e114 crush map has features 432629239337189376, adjusting msgr requires 2024-06-01T02:05:13.816 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: mon.c@-1(???).osd e114 crush map has features 432629239337189376, adjusting msgr requires 2024-06-01T02:05:13.816 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 ceph-mon[95581]: mon.c@-1(???).paxosservice(auth 1..26) refresh upgraded, format 0 -> 3 2024-06-01T02:05:13.816 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 bash[95562]: 8c572b16c86d44474cee4a9d6f0a031701d5c8fadb231e77c06ce7450e88da08 2024-06-01T02:05:13.817 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:13 smithi116 systemd[1]: Started Ceph mon.c for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:05:13.930 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:05:13 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:05:13.505Z caller=manager.go:609 level=warn component="rule manager" group=nodes msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on(instance) group_left(nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi116\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:05:14.895 INFO:teuthology.orchestra.run.smithi116.stdout:true 2024-06-01T02:05:15.155 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:14 smithi116 ceph-mon[95581]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:15.155 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:14 smithi116 ceph-mon[89502]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:15.159 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:14 smithi161 ceph-mon[26055]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:15.579 INFO:teuthology.orchestra.run.smithi116.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-01T02:05:15.579 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi116 smithi116 running 9s ago 13m - - 2024-06-01T02:05:15.580 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi161 smithi161 running 17s ago 13m - - 2024-06-01T02:05:15.580 INFO:teuthology.orchestra.run.smithi116.stdout:alertmanager.a smithi116 *:9093,9094 running (3m) 9s ago 8m 22.6M - 0.23.0 ba2b418f427c e0c297162ded 2024-06-01T02:05:15.580 INFO:teuthology.orchestra.run.smithi116.stdout:grafana.a smithi161 *:3000 running (2m) 17s ago 8m 52.4M - 8.3.5 dad864ee21e9 00b0a78ee88e 2024-06-01T02:05:15.580 INFO:teuthology.orchestra.run.smithi116.stdout:iscsi.foo.smithi116.gjuxzb smithi116 running (2m) 9s ago 6m 47.8M - 3.5 e1d6a67b021e 0e191fe3e3bb 2024-06-01T02:05:15.580 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.x smithi161 *:8443,9283,8765 running (2m) 17s ago 13m 502M - 19.0.0-2946-g3d89accf b045ac545f82 2d7fbb85e5f7 2024-06-01T02:05:15.580 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.y smithi116 *:8443,9283,8765 running (75s) 9s ago 14m 450M - 19.0.0-2946-g3d89accf b045ac545f82 bc9627c27a57 2024-06-01T02:05:15.580 INFO:teuthology.orchestra.run.smithi116.stdout:mon.a smithi116 running (29s) 9s ago 14m 41.7M 2048M 19.0.0-2946-g3d89accf b045ac545f82 b5e9b6f873ff 2024-06-01T02:05:15.580 INFO:teuthology.orchestra.run.smithi116.stdout:mon.b smithi161 running (13m) 17s ago 13m 59.6M 2048M 17.2.0 e1d6a67b021e 9011986aa6e3 2024-06-01T02:05:15.580 INFO:teuthology.orchestra.run.smithi116.stdout:mon.c smithi116 starting - - - 2048M 2024-06-01T02:05:15.580 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.a smithi116 *:9100 running (3m) 9s ago 9m 18.1M - 1.3.1 1dbe0e931976 e347748d17bb 2024-06-01T02:05:15.580 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.b smithi161 *:9100 running (2m) 17s ago 8m 17.4M - 1.3.1 1dbe0e931976 6e66182f5a1a 2024-06-01T02:05:15.580 INFO:teuthology.orchestra.run.smithi116.stdout:osd.0 smithi116 running (12m) 9s ago 12m 55.3M 4096M 17.2.0 e1d6a67b021e 0cc929b0ecad 2024-06-01T02:05:15.580 INFO:teuthology.orchestra.run.smithi116.stdout:osd.1 smithi116 running (12m) 9s ago 12m 56.8M 4096M 17.2.0 e1d6a67b021e 6ff7fc5abe58 2024-06-01T02:05:15.581 INFO:teuthology.orchestra.run.smithi116.stdout:osd.2 smithi116 running (11m) 9s ago 11m 53.4M 4096M 17.2.0 e1d6a67b021e 6aecbb10256b 2024-06-01T02:05:15.581 INFO:teuthology.orchestra.run.smithi116.stdout:osd.3 smithi116 running (11m) 9s ago 11m 55.5M 4096M 17.2.0 e1d6a67b021e a802290f5290 2024-06-01T02:05:15.581 INFO:teuthology.orchestra.run.smithi116.stdout:osd.4 smithi161 running (11m) 17s ago 11m 57.4M 4096M 17.2.0 e1d6a67b021e 70b3f89b0bd9 2024-06-01T02:05:15.581 INFO:teuthology.orchestra.run.smithi116.stdout:osd.5 smithi161 running (10m) 17s ago 10m 51.8M 4096M 17.2.0 e1d6a67b021e 68358b78a25d 2024-06-01T02:05:15.581 INFO:teuthology.orchestra.run.smithi116.stdout:osd.6 smithi161 running (10m) 17s ago 10m 52.2M 4096M 17.2.0 e1d6a67b021e 50abeb0551e7 2024-06-01T02:05:15.581 INFO:teuthology.orchestra.run.smithi116.stdout:osd.7 smithi161 running (9m) 17s ago 9m 55.1M 4096M 17.2.0 e1d6a67b021e 6004e6983735 2024-06-01T02:05:15.581 INFO:teuthology.orchestra.run.smithi116.stdout:prometheus.a smithi161 *:9095 running (113s) 17s ago 9m 65.3M - 2.33.4 514e6a882f6e e977f6ca1d73 2024-06-01T02:05:15.581 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi116.vzjrxj smithi116 *:8000 running (7m) 9s ago 7m 75.8M - 17.2.0 e1d6a67b021e df4fe3c1dbc3 2024-06-01T02:05:15.581 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi161.ashqxz smithi161 *:8000 running (7m) 17s ago 7m 76.7M - 17.2.0 e1d6a67b021e 3eecc1f3e2d3 2024-06-01T02:05:15.581 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi116.hokbmf smithi116 *:80 running (7m) 9s ago 7m 76.2M - 17.2.0 e1d6a67b021e 10950ed75dbd 2024-06-01T02:05:15.581 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi161.axjlzw smithi161 *:80 running (7m) 17s ago 7m 75.4M - 17.2.0 e1d6a67b021e 0b5d5ce47772 2024-06-01T02:05:16.003 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:05:16.003 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": { 2024-06-01T02:05:16.003 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 1, 2024-06-01T02:05:16.003 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-01T02:05:16.003 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:05:16.003 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": { 2024-06-01T02:05:16.003 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-01T02:05:16.003 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:05:16.003 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": { 2024-06-01T02:05:16.003 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2024-06-01T02:05:16.003 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:05:16.004 INFO:teuthology.orchestra.run.smithi116.stdout: "rgw": { 2024-06-01T02:05:16.004 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-06-01T02:05:16.004 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:05:16.004 INFO:teuthology.orchestra.run.smithi116.stdout: "overall": { 2024-06-01T02:05:16.004 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 13, 2024-06-01T02:05:16.004 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 4 2024-06-01T02:05:16.004 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T02:05:16.004 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:05:16.099 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:05:16.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:16.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: mon.c calling monitor election 2024-06-01T02:05:16.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: pgmap v25: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:16.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: mon.a calling monitor election 2024-06-01T02:05:16.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-01T02:05:16.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: monmap epoch 3 2024-06-01T02:05:16.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T02:05:16.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: last_changed 2024-06-01T01:51:54.507771+0000 2024-06-01T02:05:16.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: created 2024-06-01T01:50:20.260187+0000 2024-06-01T02:05:16.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: min_mon_release 17 (quincy) 2024-06-01T02:05:16.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: election_strategy: 1 2024-06-01T02:05:16.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: 0: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.a 2024-06-01T02:05:16.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: 1: [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] mon.c 2024-06-01T02:05:16.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: 2: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-06-01T02:05:16.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: fsmap 2024-06-01T02:05:16.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: osdmap e114: 8 total, 8 up, 8 in 2024-06-01T02:05:16.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: mgrmap e44: x(active, since 44s), standbys: y 2024-06-01T02:05:16.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: overall HEALTH_OK 2024-06-01T02:05:16.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:16.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:16.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='client.44100 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:05:16.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:16.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:16.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:16.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:16.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:16.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:16.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:16.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:16.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:16.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:16.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:16.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:16.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:16.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:16.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:16.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:16.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:16.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:16.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:16.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:16.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:16.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:16.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:16.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:16.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:16.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:16.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:16.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:16.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:16.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:16.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:16.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:16.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:16.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:16.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:16.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:16.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:16.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:16.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:15 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:05:16.123 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:05:16.123 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:16.123 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: mon.c calling monitor election 2024-06-01T02:05:16.123 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: pgmap v25: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:16.123 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: mon.a calling monitor election 2024-06-01T02:05:16.123 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-01T02:05:16.123 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: monmap epoch 3 2024-06-01T02:05:16.123 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T02:05:16.123 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: last_changed 2024-06-01T01:51:54.507771+0000 2024-06-01T02:05:16.123 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: created 2024-06-01T01:50:20.260187+0000 2024-06-01T02:05:16.123 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: min_mon_release 17 (quincy) 2024-06-01T02:05:16.124 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: election_strategy: 1 2024-06-01T02:05:16.124 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: 0: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.a 2024-06-01T02:05:16.124 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: 1: [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] mon.c 2024-06-01T02:05:16.124 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: 2: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-06-01T02:05:16.124 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: fsmap 2024-06-01T02:05:16.124 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: osdmap e114: 8 total, 8 up, 8 in 2024-06-01T02:05:16.124 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: mgrmap e44: x(active, since 44s), standbys: y 2024-06-01T02:05:16.124 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: overall HEALTH_OK 2024-06-01T02:05:16.124 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:16.124 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:16.124 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='client.44100 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:05:16.124 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:16.124 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:16.124 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:16.124 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:16.125 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:16.125 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:16.125 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:16.125 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:16.125 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:16.125 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:16.125 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:16.125 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:16.125 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:16.125 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:16.126 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:05:16.126 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:16.126 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: mon.c calling monitor election 2024-06-01T02:05:16.126 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: pgmap v25: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:16.126 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: mon.a calling monitor election 2024-06-01T02:05:16.126 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-01T02:05:16.126 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: monmap epoch 3 2024-06-01T02:05:16.126 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T02:05:16.126 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: last_changed 2024-06-01T01:51:54.507771+0000 2024-06-01T02:05:16.126 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: created 2024-06-01T01:50:20.260187+0000 2024-06-01T02:05:16.126 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: min_mon_release 17 (quincy) 2024-06-01T02:05:16.127 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: election_strategy: 1 2024-06-01T02:05:16.127 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: 0: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.a 2024-06-01T02:05:16.127 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: 1: [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] mon.c 2024-06-01T02:05:16.127 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: 2: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-06-01T02:05:16.127 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: fsmap 2024-06-01T02:05:16.127 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: osdmap e114: 8 total, 8 up, 8 in 2024-06-01T02:05:16.127 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: mgrmap e44: x(active, since 44s), standbys: y 2024-06-01T02:05:16.127 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: overall HEALTH_OK 2024-06-01T02:05:16.127 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:16.127 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:16.127 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='client.44100 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:05:16.127 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:16.127 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:16.127 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:16.128 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:16.128 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:16.128 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:16.128 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:16.128 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:16.128 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:16.128 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:16.128 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:16.128 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:16.128 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:16.128 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:16.128 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:16.129 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:16.129 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:16.129 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:16.129 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:16.129 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:16.129 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:16.129 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:16.129 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:16.129 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:16.129 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:16.129 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:16.129 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:16.129 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:16.130 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:16.130 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:16.130 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:16.130 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:16.130 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:16.130 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:16.130 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:16.130 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:16.130 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:16.130 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:16.130 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:05:16.130 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:16.131 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:16.131 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:16.131 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:16.131 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:16.131 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:16.131 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:16.131 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:16.131 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:16.131 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:16.131 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:16.131 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:16.131 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:16.132 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:16.132 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:16.132 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:16.132 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:16.132 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:16.132 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:16.132 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:16.132 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:16.132 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:16.132 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:16.132 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:16.132 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:05:16.366 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:05:16.366 INFO:teuthology.orchestra.run.smithi116.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-01T02:05:16.366 INFO:teuthology.orchestra.run.smithi116.stdout: "in_progress": true, 2024-06-01T02:05:16.366 INFO:teuthology.orchestra.run.smithi116.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-01T02:05:16.366 INFO:teuthology.orchestra.run.smithi116.stdout: "services_complete": [ 2024-06-01T02:05:16.366 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr" 2024-06-01T02:05:16.366 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-06-01T02:05:16.366 INFO:teuthology.orchestra.run.smithi116.stdout: "progress": "3/23 daemons upgraded", 2024-06-01T02:05:16.367 INFO:teuthology.orchestra.run.smithi116.stdout: "message": "Currently upgrading mon daemons", 2024-06-01T02:05:16.367 INFO:teuthology.orchestra.run.smithi116.stdout: "is_paused": false 2024-06-01T02:05:16.367 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:05:16.815 INFO:teuthology.orchestra.run.smithi116.stdout:HEALTH_OK 2024-06-01T02:05:17.010 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:17.011 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:17.011 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:17.011 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='client.44106 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:05:17.011 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:17.011 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:17.011 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:17.011 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='client.44112 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:05:17.011 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:17.011 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:17.011 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:17.011 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:17.011 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:17.012 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:17.012 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:17.012 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='client.? 172.21.15.116:0/3951310377' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:17.012 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:17.012 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:17.012 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:17.012 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:17.012 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:17.012 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:17.012 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:17.012 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:17.012 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:17.012 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:17.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:17.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:17.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:17.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:17.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:17.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:17.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:17.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:17.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:17.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:17.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:17.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:17.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:17.014 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:17.014 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:17.014 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:17.014 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:17.014 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:17.014 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:17.014 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:17.014 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[95581]: from='client.? 172.21.15.116:0/2963803169' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:05:17.014 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:17.015 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:17.015 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:17.015 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='client.44106 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:05:17.015 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:17.015 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:17.015 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:17.015 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='client.44112 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:05:17.015 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:17.015 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:17.015 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:17.015 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:17.015 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:17.015 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:17.016 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:17.016 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/3951310377' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:17.016 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:17.016 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:17.016 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:17.016 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:17.016 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:17.016 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:17.016 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:17.016 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:17.016 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:17.016 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:17.016 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:17.016 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:17.017 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:17.017 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:17.017 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:17.017 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:17.017 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:17.017 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:17.017 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:17.017 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:17.017 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:17.017 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:17.017 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:17.017 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:17.017 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:17.017 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:17.018 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:17.018 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:17.018 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:17.018 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:17.018 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:16 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/2963803169' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:05:17.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:05:16] "GET /metrics HTTP/1.1" 200 38136 "" "Prometheus/2.33.4" 2024-06-01T02:05:17.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:17.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:17.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:17.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='client.44106 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:05:17.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:17.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:17.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:17.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='client.44112 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:05:17.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:17.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:17.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:17.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:17.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:17.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:17.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:17.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/3951310377' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:17.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:17.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:17.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:17.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:17.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:17.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:17.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:17.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:17.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:17.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:17.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:17.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:17.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:17.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:17.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:17.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:17.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:17.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:17.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:17.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:17.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:17.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:17.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:17.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:17.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:17.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:17.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:17.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:17.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:17.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:17.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:16 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/2963803169' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:05:18.031 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:18.031 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:18.031 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:18.031 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:18.031 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='client.44124 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:05:18.031 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:18.031 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:18.031 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:18.032 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:18.032 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:18.032 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:18.032 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:18.032 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:18.032 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.032 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:18.032 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:18.032 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:18.032 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:18.032 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:18.032 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.032 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:18.033 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:18.033 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:18.033 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:18.033 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.033 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:18.033 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:18.033 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:18.033 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:18.033 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.033 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:18.034 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:18.034 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:18.034 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:18.034 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='client.44124 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:05:18.034 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:18.034 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:18.034 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:18.034 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:18.034 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:18.034 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:18.034 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:18.034 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:18.034 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.034 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:18.035 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:18.035 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:18.035 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:18.035 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:18.035 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.035 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:18.035 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:18.035 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:18.035 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:18.035 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.035 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:18.035 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:18.035 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:18.036 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:18.036 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.036 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:18.036 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:18.036 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:18.036 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:18.037 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.037 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:18.037 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:18.037 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:18.037 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:18.037 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.037 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:18.037 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:18.037 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:18.037 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:18.038 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.038 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:18.038 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:18.038 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:18.038 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:18.038 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.038 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:18.038 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:18.038 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:18.038 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:18.038 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.038 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:18.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:18.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:18.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:18.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:18.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:18.040 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:18.040 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:18.040 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.040 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:18.040 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:18.040 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:18.040 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:18.040 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:17 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:18.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:18.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:18.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:18.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='client.44124 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:05:18.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:18.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:18.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:18.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:18.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:18.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:18.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:18.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:18.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:18.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:18.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:18.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:18.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:18.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:18.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:18.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:18.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:18.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:18.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:18.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:18.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:18.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:18.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:18.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:18.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:18.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:18.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:18.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:18.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:18.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:18.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:18.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:18.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:18.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:18.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:18.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:18.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:18.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:17 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:18.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:18.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:18.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:18.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:18.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:18.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:18.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:18.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:18.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:18.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:18.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:18.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:18.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:18.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:18.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.861 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:18.861 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:18.861 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:18.861 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:18.861 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.861 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:18.861 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:18.861 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:18.861 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:18.861 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.861 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:18.862 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:05:18.660Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"172.21.15.161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"smithi161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:05:18.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:18.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:18.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:18.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:18.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:18.863 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:18.863 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:18.863 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.863 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:18.863 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:18.863 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:18.863 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:18.863 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:18.863 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:05:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:05:18.554+0000 7fcedaabb700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-06-01T02:05:19.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:19.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:19.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:19.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:19.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:19.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:19.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:19.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:19.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:19.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:19.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:19.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:19.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:19.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:19.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:19.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:19.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:19.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:19.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:19.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:19.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:19.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:19.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:19.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:19.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:19.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:19.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:19.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:19.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:19.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:19.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:19.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:19.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:19.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:19.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:19.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:19.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:19.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:19.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:19.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:19.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:19.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:19.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:19.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:19.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:19.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:19.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:19.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:19.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:19.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:19.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:19.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:19.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:19.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:19.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:19.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:19.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:19.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:19.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:19.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:19.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:19.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:19.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:19.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:19.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:19.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:19.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:19.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:19.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:19.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:19.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:19.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:19.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:19.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:19.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:19.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:19.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:19.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:19.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:19.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:19.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:19.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:19.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:19.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:19.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:19.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:19.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:19.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:19.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:19.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:18 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:20.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:20.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:20.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:20.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:20.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:20.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:20.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:20.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:20.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:20.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:20.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:20.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:20.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:20.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:20.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:20.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:20.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:20.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:20.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:20.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:20.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:20.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:20.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:20.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:20.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:20.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:20.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:20.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:20.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:20.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:20.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:20.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:20.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:20.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:20.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:20.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:20.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:20.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:20.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:20.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:20.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:20.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:20.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:20.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:20.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:20.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:20.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:20.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:20.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:20.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:20.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:20.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:20.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:20.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:20.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:20.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:20.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:20.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:19 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:20.232 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:20.232 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:20.232 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:20.232 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:20.232 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:20.232 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:20.232 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:20.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:20.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:20.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:20.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:20.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:20.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:20.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:20.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:20.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:20.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:20.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:20.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:20.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:20.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:20.234 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:20.234 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:20.234 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:20.234 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:20.234 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:20.234 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:20.234 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:20.234 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:20.234 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:20.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:20.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:20.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:20.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:20.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:20.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:20.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:20.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:20.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:20.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:20.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:20.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:20.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:20.236 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:20.236 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:20.236 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:20.236 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:20.236 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:20.236 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:20.236 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:20.236 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:20.236 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:20.236 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:20.236 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:20.236 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:20.236 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:20.236 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:20.237 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:20.237 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:20.237 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:20.237 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:20.237 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:20.237 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:20.237 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:20.237 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:20.237 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:20.237 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:20.237 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:20.237 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:20.237 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:20.238 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:20.238 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:20.238 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:20.238 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:20.238 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:20.238 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:20.238 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:20.238 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:20.238 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:20.238 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:20.238 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:20.238 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:20.238 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:20.238 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:20.239 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:20.239 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:20.239 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:20.239 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:20.239 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:20.239 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:20.239 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:20.239 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:20.239 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:20.239 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:20.239 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:20.240 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:20.240 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:20.240 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:20.240 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:20.240 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:20.240 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:20.240 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:20.240 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:20.240 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:20.240 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:20.240 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:20.240 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:20.240 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:20.240 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:20.241 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:20.241 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:20.241 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:20.241 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:20.241 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:20.241 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:20.241 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:19 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.003 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.004 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.004 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.004 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.004 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.004 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.004 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.004 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.004 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.004 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.004 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.004 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.004 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.006 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.006 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.006 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.006 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.006 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.006 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.006 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.006 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.006 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.010 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.010 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.010 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.010 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.010 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.010 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.010 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.010 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.010 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.010 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.010 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.010 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:20 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:20 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.923 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.923 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.923 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.923 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.923 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.923 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.923 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.924 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.924 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.924 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: pgmap v28: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:21.924 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.924 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.924 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.924 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.924 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.924 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.924 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.924 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.924 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.924 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.924 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.925 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.925 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.925 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.925 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.925 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.925 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.925 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.925 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.925 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.925 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.926 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.926 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.926 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.926 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.926 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.926 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.926 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.926 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.926 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.926 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: pgmap v28: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:21.926 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:21.926 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.926 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.927 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.927 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.927 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.927 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.927 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.927 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.927 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.927 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.927 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.927 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.927 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.927 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.927 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.927 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.928 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.928 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.928 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.928 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.928 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.928 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.928 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.928 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.928 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.928 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.929 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.929 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.929 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.929 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.929 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.929 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.929 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.929 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.929 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.929 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.930 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.930 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.930 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.930 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.930 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.930 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.930 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.930 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.930 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.930 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.930 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.930 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.931 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.931 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.931 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.931 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.931 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.931 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.931 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.932 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.932 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.932 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.932 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.932 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.932 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.932 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.932 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.932 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.932 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.932 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.932 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.932 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.932 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.933 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.933 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.933 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.933 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.933 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:21.933 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:21.933 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:21.933 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.933 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:21.933 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:21.933 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:21 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:22.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:22.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:22.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:22.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:22.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:22.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:22.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:22.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:22.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:22.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: pgmap v28: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:22.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:22.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:22.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:22.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:22.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:22.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:22.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:22.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:22.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:22.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:22.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:22.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:22.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:22.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:22.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:22.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:22.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:22.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:22.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:22.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:22.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:22.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:22.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:22.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:22.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:22.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:22.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:22.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:22.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:22.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:22.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:22.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:22.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:22.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:22.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:22.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:22.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:22.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:22.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:22.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:22.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:22.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:22.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:22.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:22.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:22.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:22.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:22.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:22.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:22.200 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:22.200 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:21 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:23.002 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:23.002 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:23.002 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:23.002 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:23.002 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:23.002 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:23.002 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:23.002 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:23.002 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:23.002 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:05:23.003 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:23.003 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:23.003 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:23.003 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:23.003 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:23.003 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:23.003 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:23.003 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:23.003 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:23.003 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:23.003 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:23.003 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:23.003 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:23.003 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:23.004 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:23.004 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:23.004 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:23.004 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:23.004 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:23.004 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:23.004 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:23.004 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:23.004 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:23.004 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:23.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:23.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:23.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:23.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:23.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:23.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:23.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:05:23.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:23.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:23.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:23.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:23.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:23.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:23.005 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:23.006 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:23.006 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:23.006 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:23.006 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:23.006 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:23.006 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:23.006 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:23.006 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:23.006 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:23.006 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:23.006 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:23.006 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:23.006 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:23.006 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:23.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:23.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:23.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:23.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:23.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:23.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:23.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:23.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:23.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:23.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:23.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:23.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:23.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:23.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:23.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:23.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:23.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:23.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:23.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:23.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:23.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:23.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:23.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:23.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:23.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:23.009 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:23.009 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:23.009 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:23.009 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:23.009 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:23.009 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:23.009 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:23.009 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:23.009 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:23.009 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:23.009 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:23.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:23.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:23.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:23.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:23.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:23.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:23.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:23.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:23.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:23.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:23.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:23.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:23.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:23.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:22 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:23.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:23.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:23.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:23.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:23.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:23.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:23.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:23.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:23.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:23.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:05:23.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:23.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:23.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:23.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:23.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:23.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:23.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:23.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:23.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:23.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:23.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:23.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:23.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:23.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:23.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:23.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:23.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:23.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:23.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:23.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:23.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:23.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:23.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:23.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:23.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:23.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:23.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:23.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:23.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:23.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:23.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:23.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:23.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:23.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:23.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:23.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:23.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:23.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:23.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:23.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:23.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:23.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:23.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:23.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:23.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:23.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:22 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:23.694 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:05:23.505Z caller=manager.go:609 level=warn component="rule manager" group=nodes msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on(instance) group_left(nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi116\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:05:24.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:24.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:24.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:24.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:24.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:24.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:24.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:24.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:24.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:24.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:24.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:24.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:24.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:24.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:24.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:24.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:24.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:24.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:24.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:24.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:24.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:24.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:24.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:24.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:24.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:24.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:24.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:24.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:24.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:24.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:24.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:24.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:24.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:24.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:24.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:24.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:24.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:24.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:24.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:24.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:24.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:24.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:24.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:24.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:24.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:24.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:24.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:24.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:24.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:24.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:24.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:24.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:24.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:24.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:24.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:24.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:24.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:24.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:24.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:24.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:24.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:24.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:24.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:24.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:24.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:24.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:24.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:24.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:24.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:24.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:24.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:24.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:24.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:24.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:24.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:24.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:24.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:24.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:24.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:24.055 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:24.055 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:24.055 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:24.055 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:24.055 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:24.055 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:24.055 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:24.055 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:24.055 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:24.055 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:24.055 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:24.055 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:24.056 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:24.056 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:24.056 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:24.056 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:24.056 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:24.056 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:24.056 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:24.056 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:24.056 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:23 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:24.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:24.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:24.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:24.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:24.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:24.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:24.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:24.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:24.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:24.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:24.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:24.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:24.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:24.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:24.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:24.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:24.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:24.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:24.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:24.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:24.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:24.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:24.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:24.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:24.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:24.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:24.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:23 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:25.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:25.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:25.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:25.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:25.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:25.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:25.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:25.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:25.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:25.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:25.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:25.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:25.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:25.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:25.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:25.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:25.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:25.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:25.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:25.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:25.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:25.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:25.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:25.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:25.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:25.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:25.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:25.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:25.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:25.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:25.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:25.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:25.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:25.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:25.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:25.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:25.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:25.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:25.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:25.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:25.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:25.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:25.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:25.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:25.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:25.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:25.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:25.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:25.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:25.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:25.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:25.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:25.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:25.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:25.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:25.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:25.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:25.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:25.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:25.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:25.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:25.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:25.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:25.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:25.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:25.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:25.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:25.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:25.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:25.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:25.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:25.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:25.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:25.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:25.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:25.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:25.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:25.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:25.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:25.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:25.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:25.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:25.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:25.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:25.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:25.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:25.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:25.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:25.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:25.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:25.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:25.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:25.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:25.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:25.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:25.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:24 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:25.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:25.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:25.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:25.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:25.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:25.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:25.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:25.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:25.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:25.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:25.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:25.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:25.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:25.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:25.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:25.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:25.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:25.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:25.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:25.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:25.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:25.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:25.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:25.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:25.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:25.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:25.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:25.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:25.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:25.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:25.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:25.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:25.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:25.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:25.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:25.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:25.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:25.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:25.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:25.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:25.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:25.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:25.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:25.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:25.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:25.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:25.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:25.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:24 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:25.874 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:25.874 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:25.874 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:25.874 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:25.875 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:25.875 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:25.875 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:25.875 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: pgmap v30: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:25.875 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:26.130 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:26.130 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:26.130 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:26.130 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:26.130 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:26.130 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:26.130 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:26.130 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:26.130 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:26.130 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:26.130 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:26.131 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:26.131 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:26.131 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:26.131 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:26.131 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:26.131 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:26.131 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:26.131 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:26.131 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:26.131 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:26.131 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:26.131 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:26.131 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:26.132 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:26.132 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:26.132 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:26.132 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:26.132 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:26.132 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:26.132 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:26.132 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:26.132 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:26.132 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:26.133 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:26.133 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: pgmap v30: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:26.133 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:26.133 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:26.133 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:26.133 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:26.133 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:26.133 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:26.133 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:26.133 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:26.133 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:26.133 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:26.133 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:26.133 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:26.134 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:26.134 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:26.134 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:26.134 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:26.134 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:26.134 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:26.134 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:26.134 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:26.134 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:26.134 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:26.134 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:26.135 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:26.135 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:26.135 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:26.135 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:26.135 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:26.135 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:26.135 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:26.135 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:26.135 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:26.135 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:26.135 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:26.136 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:26.136 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:26.136 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:26.136 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:26.136 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:26.136 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:26.136 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:26.136 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:26.136 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:26.136 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:26.136 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:26.137 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:26.137 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:26.137 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:26.137 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:26.137 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:26.137 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:26.137 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:26.137 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:25 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:26.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:26.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:26.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:26.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:26.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:26.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:26.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:26.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: pgmap v30: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:26.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:26.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:26.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:26.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:26.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:26.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:26.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:26.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:26.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:26.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:26.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:26.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:26.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:26.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:26.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:26.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:26.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:26.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:26.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:26.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:26.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:26.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:26.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:26.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:26.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:26.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:26.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:26.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:26.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:26.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:26.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:26.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:26.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:26.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:26.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:26.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:26.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:26.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:26.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:26.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:26.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:25 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:27.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:27.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:27.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:27.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:27.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:27.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:27.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:27.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:27.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:27.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:27.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:27.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:27.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:27.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:27.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:27.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:27.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:27.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:27.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:27.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:27.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:27.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:27.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:27.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:27.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:27.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:27.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:27.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:27.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-06-01T02:05:27.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon ok-to-stop", "ids": ["c"]}]: dispatch 2024-06-01T02:05:27.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:27.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:27.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:27.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:27.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:27.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:27.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:27.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:27.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:27.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:27.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:27.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:27.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:27.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:27.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:27.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:27.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:27.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:27.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:27.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:27.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:27.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:27.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:27.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:27.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:27.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:27.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:27.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:27.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-06-01T02:05:27.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:26 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon ok-to-stop", "ids": ["c"]}]: dispatch 2024-06-01T02:05:27.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:05:26 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:05:26] "GET /metrics HTTP/1.1" 200 38136 "" "Prometheus/2.33.4" 2024-06-01T02:05:27.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:26 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:27.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:26 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:27.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:26 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:27.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:26 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:27.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:26 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:27.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:26 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:27.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:26 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:27.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:26 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:27.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:26 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:27.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:26 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:27.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:26 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:27.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:26 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:27.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:26 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:27.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:26 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:27.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:26 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:27.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:26 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:27.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:26 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:27.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:26 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:27.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:26 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:27.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:26 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:27.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:26 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:27.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:26 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:27.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:26 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:27.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:26 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:27.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:26 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:27.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:26 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:27.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:26 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:27.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:26 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:27.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:26 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-06-01T02:05:27.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:26 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon ok-to-stop", "ids": ["c"]}]: dispatch 2024-06-01T02:05:27.972 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:27.972 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:27.973 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:27.973 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:27 smithi116 ceph-mon[89502]: Upgrade: It appears safe to stop mon.c 2024-06-01T02:05:27.973 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:27 smithi116 ceph-mon[89502]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:27.973 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:27 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:27.973 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:27 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:27.973 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:27 smithi116 ceph-mon[95581]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:27.973 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:27 smithi116 ceph-mon[95581]: Upgrade: It appears safe to stop mon.c 2024-06-01T02:05:27.973 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:27 smithi116 ceph-mon[95581]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:28.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:27 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:28.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:27 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:28.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:27 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:28.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:27 smithi161 ceph-mon[26055]: Upgrade: It appears safe to stop mon.c 2024-06-01T02:05:28.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:27 smithi161 ceph-mon[26055]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:28.944 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:05:28 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:05:28.660Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"172.21.15.161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"smithi161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:05:30.147 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:29 smithi116 ceph-mon[89502]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:30.148 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T02:05:30.148 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T02:05:30.148 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:30.148 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:29 smithi116 ceph-mon[95581]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:30.148 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:29 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T02:05:30.148 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:29 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T02:05:30.148 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:29 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:30.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:29 smithi161 ceph-mon[26055]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:30.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:29 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T02:05:30.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:29 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T02:05:30.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:29 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:30.953 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:30 smithi116 ceph-mon[89502]: Upgrade: Updating mon.c 2024-06-01T02:05:30.954 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:30 smithi116 ceph-mon[89502]: Deploying daemon mon.c on smithi116 2024-06-01T02:05:30.954 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:30 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:30.954 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:30 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:30.954 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:30 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:30.954 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:05:30.954 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:30 smithi116 ceph-mon[95581]: Upgrade: Updating mon.c 2024-06-01T02:05:30.954 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:30 smithi116 ceph-mon[95581]: Deploying daemon mon.c on smithi116 2024-06-01T02:05:30.954 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:30 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:30.955 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:30 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:30.955 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:30 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:30.955 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:30 smithi116 ceph-mon[95581]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:05:31.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:30 smithi161 ceph-mon[26055]: Upgrade: Updating mon.c 2024-06-01T02:05:31.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:30 smithi161 ceph-mon[26055]: Deploying daemon mon.c on smithi116 2024-06-01T02:05:31.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:30 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:31.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:30 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:31.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:30 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:31.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:30 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:05:32.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:31 smithi161 ceph-mon[26055]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:32.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:31 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:32.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:31 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:32.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:31 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:32.206 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:31 smithi116 ceph-mon[89502]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:32.206 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:31 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:32.206 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:31 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:32.206 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:31 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:32.206 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:31 smithi116 ceph-mon[95581]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:32.206 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:31 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:32.206 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:31 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:32.207 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:31 smithi116 ceph-mon[95581]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:33.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:32 smithi161 ceph-mon[26055]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:05:33.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:32 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:05:33.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:32 smithi116 ceph-mon[95581]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:05:33.789 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:05:33 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:05:33.504Z caller=manager.go:609 level=warn component="rule manager" group=nodes msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on(instance) group_left(nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi116\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:05:33.974 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:33 smithi116 ceph-mon[89502]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:33.975 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:33 smithi116 systemd[1]: Stopping Ceph mon.c for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:05:33.975 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:33 smithi116 ceph-mon[95581]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:33.975 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:33 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-c[95577]: 2024-06-01T02:05:33.958+0000 7f94ec013700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-01T02:05:33.975 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:33 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-c[95577]: 2024-06-01T02:05:33.958+0000 7f94ec013700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-06-01T02:05:34.177 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:33 smithi161 ceph-mon[26055]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:34.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:33 smithi116 podman[100445]: 2024-06-01 02:05:33.978993391 +0000 UTC m=+0.125054624 container died 8c572b16c86d44474cee4a9d6f0a031701d5c8fadb231e77c06ce7450e88da08 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-c, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-06-01T02:05:34.629 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:34 smithi116 podman[100445]: 2024-06-01 02:05:34.424741771 +0000 UTC m=+0.570802997 container cleanup 8c572b16c86d44474cee4a9d6f0a031701d5c8fadb231e77c06ce7450e88da08 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-c, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc) 2024-06-01T02:05:34.630 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:34 smithi116 bash[100445]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-c 2024-06-01T02:05:34.955 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:34 smithi116 podman[100458]: 2024-06-01 02:05:34.634306557 +0000 UTC m=+0.652196734 container remove 8c572b16c86d44474cee4a9d6f0a031701d5c8fadb231e77c06ce7450e88da08 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-c, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-06-01T02:05:34.955 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:34 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mon.c.service: Deactivated successfully. 2024-06-01T02:05:34.955 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:34 smithi116 systemd[1]: Stopped Ceph mon.c for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:05:34.955 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:34 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mon.c.service: Consumed 1.252s CPU time. 2024-06-01T02:05:35.276 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 systemd[1]: Starting Ceph mon.c for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:05:35.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 podman[100587]: 2024-06-01 02:05:35.183671463 +0000 UTC m=+0.016810931 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:05:35.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 podman[100587]: 2024-06-01 02:05:35.319538348 +0000 UTC m=+0.152677844 container create bd0fb0e603fa7534f39c8dab091512a8cb6bad8a3d9a396935fd05acecd63c06 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-06-01T02:05:35.839 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 podman[100587]: 2024-06-01 02:05:35.555218217 +0000 UTC m=+0.388357676 container init bd0fb0e603fa7534f39c8dab091512a8cb6bad8a3d9a396935fd05acecd63c06 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-c, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-06-01T02:05:35.840 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 podman[100587]: 2024-06-01 02:05:35.558596348 +0000 UTC m=+0.391735815 container start bd0fb0e603fa7534f39c8dab091512a8cb6bad8a3d9a396935fd05acecd63c06 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-01T02:05:35.840 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: set uid:gid to 167:167 (ceph:ceph) 2024-06-01T02:05:35.840 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc), process ceph-mon, pid 2 2024-06-01T02:05:35.840 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: pidfile_write: ignore empty --pid-file 2024-06-01T02:05:35.840 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: load: jerasure load: lrc 2024-06-01T02:05:35.840 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: RocksDB version: 7.9.2 2024-06-01T02:05:35.840 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Git sha 0 2024-06-01T02:05:35.840 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Compile date 2024-05-30 15:42:47 2024-06-01T02:05:35.840 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: DB SUMMARY 2024-06-01T02:05:35.840 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: DB Session ID: UILL81J4YYC3HJLK5O8B 2024-06-01T02:05:35.840 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: CURRENT file: CURRENT 2024-06-01T02:05:35.840 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: IDENTITY file: IDENTITY 2024-06-01T02:05:35.841 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: MANIFEST file: MANIFEST-000047 size: 277 Bytes 2024-06-01T02:05:35.841 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 2, files: 000042.sst 000045.sst 2024-06-01T02:05:35.841 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000046.log size: 1873769 ; 2024-06-01T02:05:35.841 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.error_if_exists: 0 2024-06-01T02:05:35.841 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.create_if_missing: 0 2024-06-01T02:05:35.841 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.paranoid_checks: 1 2024-06-01T02:05:35.841 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-01T02:05:35.841 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-01T02:05:35.841 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-01T02:05:35.841 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.env: 0x55f80e5dcc60 2024-06-01T02:05:35.841 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.fs: PosixFileSystem 2024-06-01T02:05:35.841 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.info_log: 0x55f80ef95b20 2024-06-01T02:05:35.841 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.max_file_opening_threads: 16 2024-06-01T02:05:35.842 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.statistics: (nil) 2024-06-01T02:05:35.842 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.use_fsync: 0 2024-06-01T02:05:35.842 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.max_log_file_size: 0 2024-06-01T02:05:35.842 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-01T02:05:35.842 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-01T02:05:35.842 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.keep_log_file_num: 1000 2024-06-01T02:05:35.842 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.recycle_log_file_num: 0 2024-06-01T02:05:35.842 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.allow_fallocate: 1 2024-06-01T02:05:35.842 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.allow_mmap_reads: 0 2024-06-01T02:05:35.842 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.allow_mmap_writes: 0 2024-06-01T02:05:35.842 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.use_direct_reads: 0 2024-06-01T02:05:35.842 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-01T02:05:35.842 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.create_missing_column_families: 0 2024-06-01T02:05:35.842 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.db_log_dir: 2024-06-01T02:05:35.843 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.wal_dir: 2024-06-01T02:05:35.843 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-01T02:05:35.843 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-01T02:05:35.843 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-01T02:05:35.843 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-01T02:05:35.843 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-01T02:05:35.843 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-01T02:05:35.843 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.advise_random_on_open: 1 2024-06-01T02:05:35.843 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.db_write_buffer_size: 0 2024-06-01T02:05:35.843 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.write_buffer_manager: 0x55f80ef66820 2024-06-01T02:05:35.843 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-01T02:05:35.843 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-01T02:05:35.843 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-01T02:05:35.844 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.rate_limiter: (nil) 2024-06-01T02:05:35.844 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-01T02:05:35.844 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.wal_recovery_mode: 2 2024-06-01T02:05:35.844 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.enable_thread_tracking: 0 2024-06-01T02:05:35.844 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.enable_pipelined_write: 0 2024-06-01T02:05:35.844 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.unordered_write: 0 2024-06-01T02:05:35.844 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-01T02:05:35.844 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-01T02:05:35.844 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-01T02:05:35.844 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-01T02:05:35.844 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.row_cache: None 2024-06-01T02:05:35.844 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.wal_filter: None 2024-06-01T02:05:35.845 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-01T02:05:35.845 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.allow_ingest_behind: 0 2024-06-01T02:05:35.846 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.two_write_queues: 0 2024-06-01T02:05:35.846 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.manual_wal_flush: 0 2024-06-01T02:05:35.846 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.wal_compression: 0 2024-06-01T02:05:35.846 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.atomic_flush: 0 2024-06-01T02:05:35.846 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-01T02:05:35.846 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-01T02:05:35.846 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-01T02:05:35.846 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.log_readahead_size: 0 2024-06-01T02:05:35.846 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-01T02:05:35.846 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.best_efforts_recovery: 0 2024-06-01T02:05:35.846 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-01T02:05:35.846 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-01T02:05:35.847 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.allow_data_in_errors: 0 2024-06-01T02:05:35.847 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.db_host_id: __hostname__ 2024-06-01T02:05:35.847 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-01T02:05:35.847 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.max_background_jobs: 2 2024-06-01T02:05:35.847 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.max_background_compactions: -1 2024-06-01T02:05:35.847 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.max_subcompactions: 1 2024-06-01T02:05:35.847 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-01T02:05:35.847 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-01T02:05:35.847 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-01T02:05:35.847 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.max_total_wal_size: 0 2024-06-01T02:05:35.847 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-01T02:05:35.847 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-01T02:05:35.847 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-01T02:05:35.847 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-01T02:05:35.848 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.max_open_files: -1 2024-06-01T02:05:35.848 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.bytes_per_sync: 0 2024-06-01T02:05:35.848 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-01T02:05:35.848 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-01T02:05:35.848 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.compaction_readahead_size: 0 2024-06-01T02:05:35.848 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.max_background_flushes: -1 2024-06-01T02:05:35.848 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Compression algorithms supported: 2024-06-01T02:05:35.848 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-01T02:05:35.848 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: kZSTD supported: 0 2024-06-01T02:05:35.848 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: kXpressCompression supported: 0 2024-06-01T02:05:35.848 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: kLZ4HCCompression supported: 1 2024-06-01T02:05:35.848 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: kZlibCompression supported: 1 2024-06-01T02:05:35.848 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: kSnappyCompression supported: 1 2024-06-01T02:05:35.848 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: kLZ4Compression supported: 1 2024-06-01T02:05:35.849 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: kBZip2Compression supported: 0 2024-06-01T02:05:35.849 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-01T02:05:35.849 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-01T02:05:35.849 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000047 2024-06-01T02:05:35.849 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-01T02:05:35.849 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-01T02:05:35.849 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.merge_operator: 2024-06-01T02:05:35.849 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.compaction_filter: None 2024-06-01T02:05:35.849 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.compaction_filter_factory: None 2024-06-01T02:05:35.849 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.sst_partitioner_factory: None 2024-06-01T02:05:35.849 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-01T02:05:35.849 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-01T02:05:35.849 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f80ef95c40) 2024-06-01T02:05:35.849 INFO:journalctl@ceph.mon.c.smithi116.stdout: cache_index_and_filter_blocks: 1 2024-06-01T02:05:35.850 INFO:journalctl@ceph.mon.c.smithi116.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-01T02:05:35.850 INFO:journalctl@ceph.mon.c.smithi116.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-01T02:05:35.850 INFO:journalctl@ceph.mon.c.smithi116.stdout: pin_top_level_index_and_filter: 1 2024-06-01T02:05:35.850 INFO:journalctl@ceph.mon.c.smithi116.stdout: index_type: 0 2024-06-01T02:05:35.850 INFO:journalctl@ceph.mon.c.smithi116.stdout: data_block_index_type: 0 2024-06-01T02:05:35.850 INFO:journalctl@ceph.mon.c.smithi116.stdout: index_shortening: 1 2024-06-01T02:05:35.850 INFO:journalctl@ceph.mon.c.smithi116.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-01T02:05:35.850 INFO:journalctl@ceph.mon.c.smithi116.stdout: checksum: 4 2024-06-01T02:05:35.850 INFO:journalctl@ceph.mon.c.smithi116.stdout: no_block_cache: 0 2024-06-01T02:05:35.850 INFO:journalctl@ceph.mon.c.smithi116.stdout: block_cache: 0x55f80ef97090 2024-06-01T02:05:35.850 INFO:journalctl@ceph.mon.c.smithi116.stdout: block_cache_name: BinnedLRUCache 2024-06-01T02:05:35.850 INFO:journalctl@ceph.mon.c.smithi116.stdout: block_cache_options: 2024-06-01T02:05:35.850 INFO:journalctl@ceph.mon.c.smithi116.stdout: capacity : 536870912 2024-06-01T02:05:35.850 INFO:journalctl@ceph.mon.c.smithi116.stdout: num_shard_bits : 4 2024-06-01T02:05:35.851 INFO:journalctl@ceph.mon.c.smithi116.stdout: strict_capacity_limit : 0 2024-06-01T02:05:35.851 INFO:journalctl@ceph.mon.c.smithi116.stdout: high_pri_pool_ratio: 0.000 2024-06-01T02:05:35.851 INFO:journalctl@ceph.mon.c.smithi116.stdout: block_cache_compressed: (nil) 2024-06-01T02:05:35.851 INFO:journalctl@ceph.mon.c.smithi116.stdout: persistent_cache: (nil) 2024-06-01T02:05:35.851 INFO:journalctl@ceph.mon.c.smithi116.stdout: block_size: 4096 2024-06-01T02:05:35.851 INFO:journalctl@ceph.mon.c.smithi116.stdout: block_size_deviation: 10 2024-06-01T02:05:35.851 INFO:journalctl@ceph.mon.c.smithi116.stdout: block_restart_interval: 16 2024-06-01T02:05:35.851 INFO:journalctl@ceph.mon.c.smithi116.stdout: index_block_restart_interval: 1 2024-06-01T02:05:35.851 INFO:journalctl@ceph.mon.c.smithi116.stdout: metadata_block_size: 4096 2024-06-01T02:05:35.851 INFO:journalctl@ceph.mon.c.smithi116.stdout: partition_filters: 0 2024-06-01T02:05:35.851 INFO:journalctl@ceph.mon.c.smithi116.stdout: use_delta_encoding: 1 2024-06-01T02:05:35.851 INFO:journalctl@ceph.mon.c.smithi116.stdout: filter_policy: bloomfilter 2024-06-01T02:05:35.851 INFO:journalctl@ceph.mon.c.smithi116.stdout: whole_key_filtering: 1 2024-06-01T02:05:35.851 INFO:journalctl@ceph.mon.c.smithi116.stdout: verify_compression: 0 2024-06-01T02:05:35.851 INFO:journalctl@ceph.mon.c.smithi116.stdout: read_amp_bytes_per_bit: 0 2024-06-01T02:05:35.852 INFO:journalctl@ceph.mon.c.smithi116.stdout: format_version: 5 2024-06-01T02:05:35.852 INFO:journalctl@ceph.mon.c.smithi116.stdout: enable_index_compression: 1 2024-06-01T02:05:35.852 INFO:journalctl@ceph.mon.c.smithi116.stdout: block_align: 0 2024-06-01T02:05:35.852 INFO:journalctl@ceph.mon.c.smithi116.stdout: max_auto_readahead_size: 262144 2024-06-01T02:05:35.852 INFO:journalctl@ceph.mon.c.smithi116.stdout: prepopulate_block_cache: 0 2024-06-01T02:05:35.852 INFO:journalctl@ceph.mon.c.smithi116.stdout: initial_auto_readahead_size: 8192 2024-06-01T02:05:35.852 INFO:journalctl@ceph.mon.c.smithi116.stdout: num_file_reads_for_auto_readahead: 2 2024-06-01T02:05:35.853 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.write_buffer_size: 33554432 2024-06-01T02:05:35.853 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.max_write_buffer_number: 2 2024-06-01T02:05:35.853 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.compression: NoCompression 2024-06-01T02:05:35.853 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.bottommost_compression: Disabled 2024-06-01T02:05:35.853 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.prefix_extractor: nullptr 2024-06-01T02:05:35.853 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-01T02:05:35.853 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.num_levels: 7 2024-06-01T02:05:35.853 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-01T02:05:35.854 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-01T02:05:35.854 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-01T02:05:35.854 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-01T02:05:35.854 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-01T02:05:35.854 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-01T02:05:35.854 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-01T02:05:35.854 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-01T02:05:35.854 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-01T02:05:35.854 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-01T02:05:35.854 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-01T02:05:35.854 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-01T02:05:35.855 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-01T02:05:35.855 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.compression_opts.level: 32767 2024-06-01T02:05:35.855 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.compression_opts.strategy: 0 2024-06-01T02:05:35.855 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-01T02:05:35.855 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-01T02:05:35.855 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-01T02:05:35.855 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-01T02:05:35.855 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.compression_opts.enabled: false 2024-06-01T02:05:35.855 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-01T02:05:35.855 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-01T02:05:35.855 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-01T02:05:35.855 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-01T02:05:35.855 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.target_file_size_base: 67108864 2024-06-01T02:05:35.856 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-01T02:05:35.856 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-01T02:05:35.856 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-01T02:05:35.856 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-01T02:05:35.856 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-01T02:05:35.856 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-01T02:05:35.856 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-01T02:05:35.856 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-01T02:05:35.856 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-01T02:05:35.856 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-01T02:05:35.856 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-01T02:05:35.856 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-01T02:05:35.856 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-01T02:05:35.857 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-01T02:05:35.857 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.arena_block_size: 1048576 2024-06-01T02:05:35.857 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-01T02:05:35.857 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-01T02:05:35.857 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.disable_auto_compactions: 0 2024-06-01T02:05:35.857 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-01T02:05:35.857 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-01T02:05:35.857 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-01T02:05:35.857 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-01T02:05:35.857 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-01T02:05:35.857 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-01T02:05:35.857 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-01T02:05:35.858 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-01T02:05:35.858 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-01T02:05:35.858 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-01T02:05:35.858 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.table_properties_collectors: 2024-06-01T02:05:35.858 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.inplace_update_support: 0 2024-06-01T02:05:35.858 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-01T02:05:35.858 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-01T02:05:35.858 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-01T02:05:35.858 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-01T02:05:35.858 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.bloom_locality: 0 2024-06-01T02:05:35.858 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.max_successive_merges: 0 2024-06-01T02:05:35.858 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-01T02:05:35.858 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.paranoid_file_checks: 0 2024-06-01T02:05:35.858 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.force_consistency_checks: 1 2024-06-01T02:05:35.859 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.report_bg_io_stats: 0 2024-06-01T02:05:35.859 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.ttl: 2592000 2024-06-01T02:05:35.859 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-01T02:05:35.859 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-01T02:05:35.859 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-01T02:05:35.859 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.enable_blob_files: false 2024-06-01T02:05:35.859 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.min_blob_size: 0 2024-06-01T02:05:35.859 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.blob_file_size: 268435456 2024-06-01T02:05:35.859 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-01T02:05:35.859 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-01T02:05:35.859 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-01T02:05:35.859 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-01T02:05:35.860 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-01T02:05:35.860 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.blob_file_starting_level: 0 2024-06-01T02:05:35.860 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-01T02:05:35.860 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 42.sst 2024-06-01T02:05:35.860 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-06-01T02:05:35.860 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000047 succeeded,manifest_file_number is 47, next_file_number is 49, last_sequence is 26347, log_number is 41,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 41 2024-06-01T02:05:35.860 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 41 2024-06-01T02:05:35.860 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b0f68023-9ac6-4086-b6ba-2926fe5078e1 2024-06-01T02:05:35.860 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717207535588829, "job": 1, "event": "recovery_started", "wal_files": [46]} 2024-06-01T02:05:35.860 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #46 mode 2 2024-06-01T02:05:35.860 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717207535594723, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 50, "file_size": 1137103, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 26352, "largest_seqno": 28370, "table_properties": {"data_size": 1130546, "index_size": 2661, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 2885, "raw_key_size": 25862, "raw_average_key_size": 22, "raw_value_size": 1112283, "raw_average_value_size": 965, "num_data_blocks": 109, "num_entries": 1152, "num_filter_entries": 1152, "num_deletions": 261, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1717207535, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b0f68023-9ac6-4086-b6ba-2926fe5078e1", "db_session_id": "UILL81J4YYC3HJLK5O8B", "orig_file_number": 50, "seqno_to_time_mapping": "N/A"}} 2024-06-01T02:05:35.860 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717207535594792, "job": 1, "event": "recovery_finished"} 2024-06-01T02:05:35.861 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: [db/version_set.cc:5047] Creating manifest 52 2024-06-01T02:05:35.861 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-06-01T02:05:35.861 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000046.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-01T02:05:35.861 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f80f086000 2024-06-01T02:05:35.861 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: DB pointer 0x55f80f070000 2024-06-01T02:05:35.861 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-01T02:05:35.861 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-01T02:05:35.861 INFO:journalctl@ceph.mon.c.smithi116.stdout: ** DB Stats ** 2024-06-01T02:05:35.861 INFO:journalctl@ceph.mon.c.smithi116.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-01T02:05:35.861 INFO:journalctl@ceph.mon.c.smithi116.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-01T02:05:35.861 INFO:journalctl@ceph.mon.c.smithi116.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-01T02:05:35.861 INFO:journalctl@ceph.mon.c.smithi116.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-01T02:05:35.861 INFO:journalctl@ceph.mon.c.smithi116.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-01T02:05:35.862 INFO:journalctl@ceph.mon.c.smithi116.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-01T02:05:35.862 INFO:journalctl@ceph.mon.c.smithi116.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-01T02:05:35.862 INFO:journalctl@ceph.mon.c.smithi116.stdout: 2024-06-01T02:05:35.862 INFO:journalctl@ceph.mon.c.smithi116.stdout: ** Compaction Stats [default] ** 2024-06-01T02:05:35.862 INFO:journalctl@ceph.mon.c.smithi116.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-01T02:05:35.862 INFO:journalctl@ceph.mon.c.smithi116.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-01T02:05:35.862 INFO:journalctl@ceph.mon.c.smithi116.stdout: L0 2/0 1.47 MB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 400.6 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-06-01T02:05:35.862 INFO:journalctl@ceph.mon.c.smithi116.stdout: L6 1/0 10.84 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-06-01T02:05:35.862 INFO:journalctl@ceph.mon.c.smithi116.stdout: Sum 3/0 12.32 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 400.6 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-06-01T02:05:35.862 INFO:journalctl@ceph.mon.c.smithi116.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 400.6 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-06-01T02:05:35.862 INFO:journalctl@ceph.mon.c.smithi116.stdout: 2024-06-01T02:05:35.862 INFO:journalctl@ceph.mon.c.smithi116.stdout: ** Compaction Stats [default] ** 2024-06-01T02:05:35.862 INFO:journalctl@ceph.mon.c.smithi116.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-01T02:05:35.862 INFO:journalctl@ceph.mon.c.smithi116.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-01T02:05:35.862 INFO:journalctl@ceph.mon.c.smithi116.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 400.6 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-06-01T02:05:35.863 INFO:journalctl@ceph.mon.c.smithi116.stdout: 2024-06-01T02:05:35.863 INFO:journalctl@ceph.mon.c.smithi116.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-01T02:05:35.863 INFO:journalctl@ceph.mon.c.smithi116.stdout: 2024-06-01T02:05:35.863 INFO:journalctl@ceph.mon.c.smithi116.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-01T02:05:35.863 INFO:journalctl@ceph.mon.c.smithi116.stdout: Flush(GB): cumulative 0.001, interval 0.001 2024-06-01T02:05:35.863 INFO:journalctl@ceph.mon.c.smithi116.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-01T02:05:35.863 INFO:journalctl@ceph.mon.c.smithi116.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-01T02:05:35.863 INFO:journalctl@ceph.mon.c.smithi116.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-01T02:05:35.863 INFO:journalctl@ceph.mon.c.smithi116.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-01T02:05:35.863 INFO:journalctl@ceph.mon.c.smithi116.stdout: Cumulative compaction: 0.00 GB write, 133.40 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-01T02:05:35.863 INFO:journalctl@ceph.mon.c.smithi116.stdout: Interval compaction: 0.00 GB write, 133.40 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-01T02:05:35.864 INFO:journalctl@ceph.mon.c.smithi116.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-01T02:05:35.864 INFO:journalctl@ceph.mon.c.smithi116.stdout: Block cache BinnedLRUCache@0x55f80ef97090#2 capacity: 512.00 MB usage: 7.53 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-06-01T02:05:35.864 INFO:journalctl@ceph.mon.c.smithi116.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,3.75 KB,0.000715256%) IndexBlock(2,3.78 KB,0.000721216%) Misc(1,0.00 KB,0%) 2024-06-01T02:05:35.864 INFO:journalctl@ceph.mon.c.smithi116.stdout: 2024-06-01T02:05:35.864 INFO:journalctl@ceph.mon.c.smithi116.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-01T02:05:35.864 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: starting mon.c rank 1 at public addrs [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] at bind addrs [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T02:05:35.864 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: mon.c@-1(???) e3 preinit fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T02:05:35.864 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: mon.c@-1(???).mds e1 new map 2024-06-01T02:05:35.864 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: mon.c@-1(???).mds e1 print_map 2024-06-01T02:05:35.864 INFO:journalctl@ceph.mon.c.smithi116.stdout: e1 2024-06-01T02:05:35.864 INFO:journalctl@ceph.mon.c.smithi116.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-01T02:05:35.864 INFO:journalctl@ceph.mon.c.smithi116.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-01T02:05:35.864 INFO:journalctl@ceph.mon.c.smithi116.stdout: legacy client fscid: -1 2024-06-01T02:05:35.864 INFO:journalctl@ceph.mon.c.smithi116.stdout: 2024-06-01T02:05:35.865 INFO:journalctl@ceph.mon.c.smithi116.stdout: No filesystems configured 2024-06-01T02:05:35.865 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: mon.c@-1(???).osd e114 crush map has features 3314933000854323200, adjusting msgr requires 2024-06-01T02:05:35.865 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: mon.c@-1(???).osd e114 crush map has features 432629239337189376, adjusting msgr requires 2024-06-01T02:05:35.865 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: mon.c@-1(???).osd e114 crush map has features 432629239337189376, adjusting msgr requires 2024-06-01T02:05:35.865 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: mon.c@-1(???).osd e114 crush map has features 432629239337189376, adjusting msgr requires 2024-06-01T02:05:35.865 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 ceph-mon[100606]: mon.c@-1(???).paxosservice(auth 1..27) refresh upgraded, format 0 -> 3 2024-06-01T02:05:35.865 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 bash[100587]: bd0fb0e603fa7534f39c8dab091512a8cb6bad8a3d9a396935fd05acecd63c06 2024-06-01T02:05:35.865 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:35 smithi116 systemd[1]: Started Ceph mon.c for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:05:36.944 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:05:36 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:05:36] "GET /metrics HTTP/1.1" 200 38268 "" "Prometheus/2.33.4" 2024-06-01T02:05:37.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:37.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: mon.c calling monitor election 2024-06-01T02:05:37.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: mon.a calling monitor election 2024-06-01T02:05:37.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-01T02:05:37.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: monmap epoch 3 2024-06-01T02:05:37.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T02:05:37.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: last_changed 2024-06-01T01:51:54.507771+0000 2024-06-01T02:05:37.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: created 2024-06-01T01:50:20.260187+0000 2024-06-01T02:05:37.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: min_mon_release 17 (quincy) 2024-06-01T02:05:37.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: election_strategy: 1 2024-06-01T02:05:37.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: 0: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.a 2024-06-01T02:05:37.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: 1: [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] mon.c 2024-06-01T02:05:37.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: 2: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-06-01T02:05:37.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: fsmap 2024-06-01T02:05:37.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: osdmap e114: 8 total, 8 up, 8 in 2024-06-01T02:05:37.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: mgrmap e44: x(active, since 65s), standbys: y 2024-06-01T02:05:37.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: overall HEALTH_OK 2024-06-01T02:05:37.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:37.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:37.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:37.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:37.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:37.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:37.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:37.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:37.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:37.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:37.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:37.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:37.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:37.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:37.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:37.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:37.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:37.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:37.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:37.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:37.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:37.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:37.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:37.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:37.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:37.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:37.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:37.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:37.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:37.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:37.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:37.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:37.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:37.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:37.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:37.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:37.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:37.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:37.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:37.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:37.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:37 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:37.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:37.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: mon.c calling monitor election 2024-06-01T02:05:37.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: mon.a calling monitor election 2024-06-01T02:05:37.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-01T02:05:37.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: monmap epoch 3 2024-06-01T02:05:37.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T02:05:37.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: last_changed 2024-06-01T01:51:54.507771+0000 2024-06-01T02:05:37.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: created 2024-06-01T01:50:20.260187+0000 2024-06-01T02:05:37.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: min_mon_release 17 (quincy) 2024-06-01T02:05:37.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: election_strategy: 1 2024-06-01T02:05:37.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: 0: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.a 2024-06-01T02:05:37.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: 1: [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] mon.c 2024-06-01T02:05:37.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: 2: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-06-01T02:05:37.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: fsmap 2024-06-01T02:05:37.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: osdmap e114: 8 total, 8 up, 8 in 2024-06-01T02:05:37.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: mgrmap e44: x(active, since 65s), standbys: y 2024-06-01T02:05:37.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: overall HEALTH_OK 2024-06-01T02:05:37.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:37.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:37.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:37.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:37.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:37.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:37.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:37.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:37.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:37.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:37.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:37.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:37.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:37.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:37.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:37.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:37.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:37.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:37.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:37.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:37.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:37.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:37.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:37.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:37.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:37.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:37.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:37.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:37.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:37.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:37.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:37.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:37.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:37.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:37.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:37.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:37.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:37.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:37.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:37.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:37.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:37.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:37.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: mon.c calling monitor election 2024-06-01T02:05:37.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: mon.a calling monitor election 2024-06-01T02:05:37.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-01T02:05:37.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: monmap epoch 3 2024-06-01T02:05:37.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T02:05:37.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: last_changed 2024-06-01T01:51:54.507771+0000 2024-06-01T02:05:37.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: created 2024-06-01T01:50:20.260187+0000 2024-06-01T02:05:37.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: min_mon_release 17 (quincy) 2024-06-01T02:05:37.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: election_strategy: 1 2024-06-01T02:05:37.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: 0: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.a 2024-06-01T02:05:37.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: 1: [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] mon.c 2024-06-01T02:05:37.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: 2: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-06-01T02:05:37.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: fsmap 2024-06-01T02:05:37.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: osdmap e114: 8 total, 8 up, 8 in 2024-06-01T02:05:37.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: mgrmap e44: x(active, since 65s), standbys: y 2024-06-01T02:05:37.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: overall HEALTH_OK 2024-06-01T02:05:37.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:37.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:37.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:37.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:37.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:37.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:37.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:37.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:37.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:37.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:37.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:37.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:37.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:37.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:37.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:37.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:37.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:37.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:37.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:37.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:37.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:37.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:37.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:37.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:37.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:37.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:37.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:37.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:37.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:37.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:37.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:37.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:37.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:37.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:37.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:37.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:37.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:37.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:37.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:37.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:37.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:37 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:38.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:38.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:38.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:38.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:38.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:38.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:38.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:38.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:38.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:38.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:38.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:38.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:38.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:38.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:38.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:38.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:38.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:38.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:38.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:38.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:38.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:38.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:38.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:38.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:38.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:38.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:38.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:38.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:38.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:38.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:38.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:38.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:38.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:38.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:38.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:38.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:38.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:38.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:38.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:38.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:38.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:38.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:38.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:38.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:38.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:38.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:38.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:38.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:38.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:38.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:38.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:38.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:38.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:38.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:38.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:38.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:38.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:38.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:38.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:38.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:38.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:38.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:38.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:38.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:38.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:38.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:38.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:38.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:38.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:38.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:38.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:38.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:38.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:38.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:38.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:38.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:38.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:38.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:38.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:38.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:38.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:38.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:38.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:38.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:38.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:38.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:38.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:38.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:38.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:38.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:38 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:38.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:38.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:38.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:38.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:38.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:38.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:38.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:38.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:38.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:38.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:38.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:38.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:38.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:38.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:38.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:38.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:38.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:38.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:38.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:38.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:38.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:38.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:38.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:38.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:38.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:38.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:38.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:38.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:38.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:38.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:38.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:38.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:38.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:38.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:38.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:38.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:38.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:38.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:38.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:38.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:38.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:38.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:38.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:38.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:38.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:38.944 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:05:38 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:05:38.660Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"172.21.15.161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"smithi161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:05:39.399 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:39.399 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:39.399 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:39.399 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:39.399 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:39.399 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:39.399 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:39.400 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:39.400 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:39.400 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:39.400 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:39.400 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:39.400 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:39.400 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:39.400 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:39.400 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:39.400 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:39.400 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:39.400 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:39.401 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:39.401 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:39.401 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:39.401 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:39.401 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:39.401 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:39.401 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:39.401 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:39.401 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:39.401 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:39.402 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:39.402 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:39.402 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:39.402 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:39.402 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:39.402 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:39.402 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:39.402 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:39.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:39.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:39.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:39.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:39.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:39.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:39.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:39.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:39.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:39.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:39.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:39.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:39.404 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:39.404 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:39.404 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:39.404 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:39.404 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:39.404 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:39.404 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:39.404 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:39.404 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:39.404 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:39.404 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:39.405 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:39.405 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:39.405 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:39.405 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:39.405 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:39.405 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:39.405 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:39.405 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:39.405 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:39.406 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:39.406 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:39.406 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:39.406 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:39.406 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:39.406 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:39.406 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:39.406 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:39.406 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:39.406 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:39.407 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:39.407 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:39.407 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:39.407 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:39.407 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:39.407 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:39.407 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:39.407 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:39.407 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:39.408 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:39.408 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:39.408 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:39.408 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:39.408 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:39.408 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:39.408 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:39.408 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:39.408 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:39.408 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:39.408 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:39.409 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:39.409 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:39.409 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:39.409 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:39.409 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:39.409 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:39.409 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:39.409 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:39.409 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:39 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:39.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:39.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:39.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:39.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:39.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:39.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:39.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:39.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:39.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:39.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:39.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:39.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:39.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:39.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:39.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:39.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:39.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:39.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:39.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:39.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:39.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:39.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:39.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:39.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:39.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:39.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:39.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:39.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:39.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:39.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:39.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:39.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:39.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:39.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:39.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:39.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:39.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:39.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:39.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:39.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:39.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:39.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:39.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:39.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:39.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:39.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:39.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:39.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:39.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:39.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:39.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:39.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:39.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:39.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:39 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:40.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:40.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:40.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:40.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:40.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:40.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:40.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:40.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:40.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:40.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:40.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:40.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:40.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:40.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:40.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:40.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:40.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:40.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:40.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:40.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:40.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:40.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:40.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:40.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:40.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:40.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:40.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:40.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:40.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:40.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:40.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:40.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:40.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:40.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:40.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:40.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:40.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:40.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:40.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:40.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:40.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:40.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:40.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:40.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:40.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:40.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:40.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:40.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:40.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:40.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:40.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:40.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:40.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:40.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:40.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:40.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:40.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:40.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:40.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:40.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:40.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:40.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:40.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:40.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:40.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:40.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:40.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:40.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:40.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:40.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:40.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:40.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:40.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:40.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:40.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:40.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:40.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:40.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:40.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:40.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:40.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:40.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:40.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:40.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:40.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:40.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:40.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:40.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:40.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:40.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:40.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:40.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:40.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:40.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:40.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:40.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:40.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:40.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:40.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:40.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:40.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:40.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:40.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:40.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:40.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:40.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:40.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:40.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:40 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:40.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:40.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:40.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:40.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:40.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:40.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:40.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:40.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:40.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:40.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:40.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:40.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:40.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:40.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:40.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:40.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:40.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:40.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:40.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:40.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:40.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:40.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:40.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:40.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:40.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:40.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:40.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:40.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:40.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:40.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:40.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:40.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:40.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:40.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:40.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:40.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:40.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:40.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:40.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:40.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:40.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:40.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:40.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:40.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:40.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:40.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:40.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:40.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:40.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:40.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:40.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:40.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:40.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:40.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:40 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:41.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:41.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:41.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:41.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:41.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:41.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:41.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:41.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:41.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:41.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:41.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:41.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:41.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:41.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:41.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:41.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:41.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:41.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:41.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:41.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:41.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:41.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:41.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:41.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:41.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:41.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:41.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:41.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:41.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:41.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:41.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:41.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:41.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:41.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:41.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:41.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:41.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:41.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:41.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:41.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:41.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:41.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:41.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:41.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:41.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:41.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:41.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:41.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:41.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:41.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:41.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:41.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:41.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:41.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:41.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:41.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:41.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:41.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:41.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:41.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:41.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:41.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:41.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:41.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:41.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:41.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:41.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:41.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:41.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:41.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:41.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:41.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:41.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:41.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:41.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:41.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:41.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:41.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:41.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:41.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:41.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:41.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:41.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:41.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:41.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:41.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:41.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:41.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:41.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:41.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:41.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:41.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:41.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:41.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:41.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:41.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:41.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:41.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:41.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:41.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:41.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:41.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:41.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:41.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:41.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:41.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:41.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:41.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:41.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:41.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:41 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:41.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:41.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:41.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:41.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:41.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:41.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:41.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:41.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:41.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:41.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:41.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:41.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:41.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:41.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:41.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:41.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:41.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:41.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:41.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:41.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:41.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:41.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:41.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:41.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:41.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:41.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:41.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:41.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:41.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:41.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:41.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:41.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:41.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:41.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:41.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:41.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:41.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:41.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:41.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:41.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:41.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:41.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:41.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:41.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:41.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:41.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:41.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:41.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:41.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:41.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:41.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:41.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:41.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:41.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:41.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:41 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:42.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:42.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:42.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:42.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:42.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:42.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:42.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:42.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:42.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:42.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:42.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:42.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:42.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.554 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:42.554 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.554 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.554 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.555 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.555 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.555 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:42.555 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.555 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.556 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.556 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.556 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.556 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:42.556 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:05:42.557 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.557 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.557 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.557 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.557 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.557 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:42.558 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.558 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.558 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.558 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.558 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.559 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.559 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.559 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.559 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.559 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.559 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.560 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.560 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.560 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.560 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.560 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.560 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.561 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.561 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.561 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.562 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.562 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.562 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:42.562 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.562 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.562 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.563 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.563 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.563 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:42.563 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:05:42.563 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.563 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.563 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.564 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.564 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.564 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:42.564 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.564 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.564 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.564 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.565 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.565 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.565 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.565 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.565 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.565 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.566 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.566 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.566 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.566 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.566 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.566 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.567 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.567 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.567 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.567 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:42 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:42.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:05:42.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:42.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:42.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:42.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:42.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:42.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:42.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:42 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:43.414 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:43.415 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:43.415 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:43.415 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:43.415 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:43.415 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:43.415 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:43.415 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:43.415 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:43.415 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:43.415 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:43.415 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:43.416 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:43.416 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:43.416 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:43.416 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:43.416 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:43.416 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:43.416 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:43.416 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:43.416 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:43.416 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:43.416 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:43.416 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:43.416 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:43.416 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:43.417 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:43.417 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:43.417 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:43.417 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:43.417 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:43.417 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:43.417 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:43.417 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:43.417 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:43.417 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:43.417 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:43.418 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:43.418 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:43.418 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:43.418 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:43.418 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:43.418 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:43.418 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:43.418 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:43.418 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:43.418 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:43.418 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:43.418 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:43.418 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:43.419 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:43.419 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:43.419 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:43.419 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:43.419 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:43.419 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:43.419 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:43.419 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:43.419 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:43.419 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:43.420 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:43.420 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:43.420 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:43.420 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:43.420 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:43.420 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:43.420 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:43.420 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:43.420 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:43.420 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:43.420 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:43.420 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:43.420 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:43.420 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:43.421 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:43.421 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:43.421 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:43.421 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:43.421 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:43.421 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:43.421 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:43.421 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:43.421 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:43.421 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:43.421 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:43.422 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:43.422 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:43.422 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:43.422 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:43.422 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:43.422 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:43.422 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:43.422 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:43.422 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:43.422 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:43.422 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:43.422 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:43.422 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:43.423 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:43.423 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:43.423 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:43.423 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:43.423 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:43.423 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:43.423 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:43.423 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:43.694 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:05:43.505Z caller=manager.go:609 level=warn component="rule manager" group=nodes msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on(instance) group_left(nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi116\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:05:43.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:43.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:43.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:43.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:43.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:43.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:43.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:43.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:43.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:43.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:43.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:43.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:43.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:43.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:43.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:43.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:43.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:43.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:43.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:43.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:43.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:43.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:43.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:43.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:43.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:43.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:43.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:43.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:43.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:43.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:43.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:43.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:43.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:43.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:43.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:43.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:43.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:43.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:43.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:43.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:43.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:43.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:43.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:43.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:43.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:43.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:43.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:43.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:43.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:43.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:43.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:43.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:43.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:43 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:44.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:44.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:44.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:44.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:44.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:44.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:44.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:44.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:44.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:44.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:44.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:44.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:44.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:44.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:44.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:44.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:44.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:44.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:44.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:44.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:44.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:44.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:44.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:44.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:44.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:44.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:44.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:44.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:44.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:44.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:44.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:44.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:44.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:44.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:44.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:44.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:44.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:44.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:44.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:44.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:44.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:44.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:44.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:44.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:44.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:44.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:44.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:44.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:44.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:44.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:44.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:44.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:44.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:44.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:44.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:44.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:44.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:44.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:44.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:44.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:44.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:44.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:44.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:44.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:44.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:44.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:44.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:44.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:44.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:44.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:44.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:44.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:44.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:44.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:44.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:44.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:44.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:44.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:44.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:44.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:44.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:44.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:44.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:44.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:44.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:44.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:44.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:44.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:44.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:44.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:44 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:44.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:44.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:44.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:44.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:44.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:44.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:44.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:44.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:44.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:44.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:44.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:44.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:44.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:44.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:44.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:44.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:44.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:44.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:44.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:44.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:44.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:44.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:44.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:44.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:44.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:44.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:44.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:44.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:44.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:44.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:44.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:44.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:44.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:44.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:44.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:44.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:44.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:44.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:44.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:44.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:44.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:44.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:44.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:44.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:44.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:44 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:45.479 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:45.479 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:45.479 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:45.479 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:45.479 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:45.479 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:45.479 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:45.479 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:45.479 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:45.479 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:45.479 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:45.479 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:45.480 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:45.480 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:45.480 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:45.480 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:45.480 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:45.480 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:45.480 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:45.480 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:45.480 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:45.480 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:45.480 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:45.480 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:45.480 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:45.481 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:45.481 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:45.481 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: pgmap v40: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:45.481 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:45.481 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:45.481 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:45.481 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:45.481 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:45.481 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:45.481 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:45.482 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:45.482 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:45.482 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:45.482 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:45.482 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:45.482 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:45.482 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:45.482 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:45.482 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:45.482 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:45.482 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:45.482 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:45.482 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:45.483 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:45.483 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:45.483 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:45.483 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:45.483 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:45.483 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:45.483 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:45.483 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:45.483 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:45.483 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: pgmap v40: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:45.483 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:45.483 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:45.483 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:45.484 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:45.484 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:45.484 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:45.484 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:45.484 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:45.484 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:45.484 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:45.484 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:45.484 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:45.484 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:45.484 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:45.484 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:45.484 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:45.484 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:45.485 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:45.485 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:45.485 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:45.485 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:45.485 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:45.485 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:45.485 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:45.485 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:45.485 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:45.485 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:45.485 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:45.485 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:45.486 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:45.486 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:45.486 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:45.486 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:45.486 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:45.486 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:45.486 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:45.486 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:45.486 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:45.486 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:45.486 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:45 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:45.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:45.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:45.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:45.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:45.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:45.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:45.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:45.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:45.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:45.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:45.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:45.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:45.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:45.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:45.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:45.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:45.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:45.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:45.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:45.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:45.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:45.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:45.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:45.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:45.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:45.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:45.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:45.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: pgmap v40: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:45.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:45.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:45.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:45.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:45.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:45.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:45.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:45.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:45.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:45.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:45.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:45.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:45.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:45.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:45.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:45.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:45.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:45.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:45.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:45.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:45.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:45 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:46.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:46.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:46.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:46.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:46.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:46.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:46.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:46.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:46.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:46.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:46.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:46.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:46.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:46.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:46.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:46.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:46.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:46.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:46.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:46.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:46.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:46.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:46.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:46.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:46.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:46.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:46.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:46.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:46.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:46.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:46.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:46.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:46.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:46.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:46.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:46.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:46.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:46.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:46.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:46.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:46.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:46.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:46.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:46.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:46.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:46.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:46.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:46.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:46.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:46.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:46.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:46.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:46.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:46.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:46.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:46.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:46.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:46.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:46.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:05:46.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:46.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:46.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:46.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:46.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:46.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:46.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:46.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:46.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:46.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:46.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-06-01T02:05:46.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon ok-to-stop", "ids": ["b"]}]: dispatch 2024-06-01T02:05:46.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:46.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:46.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:46.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:46.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:46.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:46.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:46.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:05:46.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:46.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:46.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:46.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:46.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:46.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:46.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:46.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:46.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:46.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:46.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-06-01T02:05:46.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon ok-to-stop", "ids": ["b"]}]: dispatch 2024-06-01T02:05:46.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:46.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:46.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:46.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:46.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:46.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:46.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:46.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:46.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:46.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:46.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:46.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:46.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:46.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:46.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:46.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:46.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:46.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:46.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:46.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:46.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:46.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:46.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:46.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:46.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:46.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:46.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:46.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:46.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:46.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:46.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:46.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:46.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:46.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:05:46.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:46.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:46.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:46.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:46.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:46.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:46.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:46.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:46.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:46.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:05:46.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:46.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-06-01T02:05:46.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon ok-to-stop", "ids": ["b"]}]: dispatch 2024-06-01T02:05:47.083 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:05:46 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:05:46] "GET /metrics HTTP/1.1" 200 38261 "" "Prometheus/2.33.4" 2024-06-01T02:05:47.193 INFO:teuthology.orchestra.run.smithi116.stdout:true 2024-06-01T02:05:47.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:47 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:47.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:47 smithi161 ceph-mon[26055]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:47.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:47 smithi161 ceph-mon[26055]: Upgrade: It appears safe to stop mon.b 2024-06-01T02:05:47.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:47 smithi161 ceph-mon[26055]: pgmap v41: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:47.456 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:47 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:47.456 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:47 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:47.456 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:47 smithi116 ceph-mon[100606]: Upgrade: It appears safe to stop mon.b 2024-06-01T02:05:47.456 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:47 smithi116 ceph-mon[100606]: pgmap v41: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:47.456 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:47 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:47.456 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:47 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:47.456 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:47 smithi116 ceph-mon[89502]: Upgrade: It appears safe to stop mon.b 2024-06-01T02:05:47.457 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:47 smithi116 ceph-mon[89502]: pgmap v41: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:48.006 INFO:teuthology.orchestra.run.smithi116.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-01T02:05:48.006 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi116 smithi116 running 0s ago 14m - - 2024-06-01T02:05:48.007 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi161 smithi161 running 9s ago 14m - - 2024-06-01T02:05:48.007 INFO:teuthology.orchestra.run.smithi116.stdout:alertmanager.a smithi116 *:9093,9094 running (3m) 0s ago 8m 22.6M - 0.23.0 ba2b418f427c e0c297162ded 2024-06-01T02:05:48.007 INFO:teuthology.orchestra.run.smithi116.stdout:grafana.a smithi161 *:3000 running (3m) 9s ago 8m 51.5M - 8.3.5 dad864ee21e9 00b0a78ee88e 2024-06-01T02:05:48.007 INFO:teuthology.orchestra.run.smithi116.stdout:iscsi.foo.smithi116.gjuxzb smithi116 running (2m) 0s ago 7m 50.7M - 3.5 e1d6a67b021e 0e191fe3e3bb 2024-06-01T02:05:48.007 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.x smithi161 *:8443,9283,8765 running (2m) 9s ago 13m 510M - 19.0.0-2946-g3d89accf b045ac545f82 2d7fbb85e5f7 2024-06-01T02:05:48.007 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.y smithi116 *:8443,9283,8765 running (108s) 0s ago 15m 452M - 19.0.0-2946-g3d89accf b045ac545f82 bc9627c27a57 2024-06-01T02:05:48.007 INFO:teuthology.orchestra.run.smithi116.stdout:mon.a smithi116 running (62s) 0s ago 15m 50.0M 2048M 19.0.0-2946-g3d89accf b045ac545f82 b5e9b6f873ff 2024-06-01T02:05:48.007 INFO:teuthology.orchestra.run.smithi116.stdout:mon.b smithi161 running (13m) 9s ago 13m 65.3M 2048M 17.2.0 e1d6a67b021e 9011986aa6e3 2024-06-01T02:05:48.007 INFO:teuthology.orchestra.run.smithi116.stdout:mon.c smithi116 running (12s) 0s ago 14m 35.0M 2048M 19.0.0-2946-g3d89accf b045ac545f82 bd0fb0e603fa 2024-06-01T02:05:48.007 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.a smithi116 *:9100 running (3m) 0s ago 9m 17.2M - 1.3.1 1dbe0e931976 e347748d17bb 2024-06-01T02:05:48.007 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.b smithi161 *:9100 running (3m) 9s ago 9m 17.6M - 1.3.1 1dbe0e931976 6e66182f5a1a 2024-06-01T02:05:48.007 INFO:teuthology.orchestra.run.smithi116.stdout:osd.0 smithi116 running (13m) 0s ago 13m 55.3M 4096M 17.2.0 e1d6a67b021e 0cc929b0ecad 2024-06-01T02:05:48.007 INFO:teuthology.orchestra.run.smithi116.stdout:osd.1 smithi116 running (12m) 0s ago 12m 56.7M 4096M 17.2.0 e1d6a67b021e 6ff7fc5abe58 2024-06-01T02:05:48.008 INFO:teuthology.orchestra.run.smithi116.stdout:osd.2 smithi116 running (12m) 0s ago 12m 53.9M 4096M 17.2.0 e1d6a67b021e 6aecbb10256b 2024-06-01T02:05:48.008 INFO:teuthology.orchestra.run.smithi116.stdout:osd.3 smithi116 running (12m) 0s ago 12m 55.7M 4096M 17.2.0 e1d6a67b021e a802290f5290 2024-06-01T02:05:48.008 INFO:teuthology.orchestra.run.smithi116.stdout:osd.4 smithi161 running (11m) 9s ago 11m 57.5M 4096M 17.2.0 e1d6a67b021e 70b3f89b0bd9 2024-06-01T02:05:48.008 INFO:teuthology.orchestra.run.smithi116.stdout:osd.5 smithi161 running (11m) 9s ago 11m 52.0M 4096M 17.2.0 e1d6a67b021e 68358b78a25d 2024-06-01T02:05:48.008 INFO:teuthology.orchestra.run.smithi116.stdout:osd.6 smithi161 running (10m) 9s ago 10m 52.5M 4096M 17.2.0 e1d6a67b021e 50abeb0551e7 2024-06-01T02:05:48.008 INFO:teuthology.orchestra.run.smithi116.stdout:osd.7 smithi161 running (10m) 9s ago 10m 55.5M 4096M 17.2.0 e1d6a67b021e 6004e6983735 2024-06-01T02:05:48.008 INFO:teuthology.orchestra.run.smithi116.stdout:prometheus.a smithi161 *:9095 running (2m) 9s ago 9m 65.4M - 2.33.4 514e6a882f6e e977f6ca1d73 2024-06-01T02:05:48.008 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi116.vzjrxj smithi116 *:8000 running (7m) 0s ago 7m 75.9M - 17.2.0 e1d6a67b021e df4fe3c1dbc3 2024-06-01T02:05:48.008 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi161.ashqxz smithi161 *:8000 running (7m) 9s ago 7m 76.9M - 17.2.0 e1d6a67b021e 3eecc1f3e2d3 2024-06-01T02:05:48.008 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi116.hokbmf smithi116 *:80 running (7m) 0s ago 7m 76.4M - 17.2.0 e1d6a67b021e 10950ed75dbd 2024-06-01T02:05:48.008 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi161.axjlzw smithi161 *:80 running (7m) 9s ago 7m 75.5M - 17.2.0 e1d6a67b021e 0b5d5ce47772 2024-06-01T02:05:48.444 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:05:48.444 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": { 2024-06-01T02:05:48.444 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 1, 2024-06-01T02:05:48.444 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-01T02:05:48.444 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:05:48.444 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": { 2024-06-01T02:05:48.444 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-01T02:05:48.444 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:05:48.445 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": { 2024-06-01T02:05:48.445 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2024-06-01T02:05:48.445 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:05:48.445 INFO:teuthology.orchestra.run.smithi116.stdout: "rgw": { 2024-06-01T02:05:48.445 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-06-01T02:05:48.445 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:05:48.445 INFO:teuthology.orchestra.run.smithi116.stdout: "overall": { 2024-06-01T02:05:48.445 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 13, 2024-06-01T02:05:48.445 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 4 2024-06-01T02:05:48.445 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T02:05:48.445 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:05:48.798 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:05:48.798 INFO:teuthology.orchestra.run.smithi116.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-01T02:05:48.798 INFO:teuthology.orchestra.run.smithi116.stdout: "in_progress": true, 2024-06-01T02:05:48.798 INFO:teuthology.orchestra.run.smithi116.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-01T02:05:48.798 INFO:teuthology.orchestra.run.smithi116.stdout: "services_complete": [ 2024-06-01T02:05:48.798 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr" 2024-06-01T02:05:48.798 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-06-01T02:05:48.798 INFO:teuthology.orchestra.run.smithi116.stdout: "progress": "4/23 daemons upgraded", 2024-06-01T02:05:48.798 INFO:teuthology.orchestra.run.smithi116.stdout: "message": "Currently upgrading mon daemons", 2024-06-01T02:05:48.798 INFO:teuthology.orchestra.run.smithi116.stdout: "is_paused": false 2024-06-01T02:05:48.799 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:05:48.944 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:05:48 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:05:48.660Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"172.21.15.161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"smithi161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:05:48.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:48 smithi161 ceph-mon[26055]: from='client.64102 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:05:48.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:48 smithi161 ceph-mon[26055]: from='client.44142 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:05:48.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:48 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:48.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:48 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:48.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:48 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:48.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:48 smithi161 ceph-mon[26055]: Upgrade: Updating mon.b 2024-06-01T02:05:48.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:48 smithi161 ceph-mon[26055]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:48.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:48 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T02:05:48.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:48 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T02:05:48.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:48 smithi161 ceph-mon[26055]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:48.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:48 smithi161 ceph-mon[26055]: Deploying daemon mon.b on smithi161 2024-06-01T02:05:48.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:48 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/1297720641' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:49.009 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:48 smithi116 ceph-mon[89502]: from='client.64102 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:05:49.009 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:48 smithi116 ceph-mon[89502]: from='client.44142 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:05:49.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:48 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:49.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:48 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:49.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:48 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:49.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:48 smithi116 ceph-mon[89502]: Upgrade: Updating mon.b 2024-06-01T02:05:49.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:48 smithi116 ceph-mon[89502]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:49.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T02:05:49.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T02:05:49.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:49.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:48 smithi116 ceph-mon[89502]: Deploying daemon mon.b on smithi161 2024-06-01T02:05:49.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:48 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/1297720641' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:49.011 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:48 smithi116 ceph-mon[100606]: from='client.64102 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:05:49.011 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:48 smithi116 ceph-mon[100606]: from='client.44142 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:05:49.012 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:48 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:49.012 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:48 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:49.012 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:48 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:49.012 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:48 smithi116 ceph-mon[100606]: Upgrade: Updating mon.b 2024-06-01T02:05:49.012 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:48 smithi116 ceph-mon[100606]: from='mgr.25280 ' entity='mgr.x' 2024-06-01T02:05:49.012 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T02:05:49.012 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T02:05:49.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:49.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:48 smithi116 ceph-mon[100606]: Deploying daemon mon.b on smithi161 2024-06-01T02:05:49.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:48 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/1297720641' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:05:49.231 INFO:teuthology.orchestra.run.smithi116.stdout:HEALTH_OK 2024-06-01T02:05:49.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:49 smithi161 systemd[1]: Stopping Ceph mon.b for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:05:49.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:49 smithi161 ceph-mon[26055]: from='client.64114 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:05:49.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:49 smithi161 ceph-mon[26055]: pgmap v42: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:49.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:49 smithi161 ceph-mon[26055]: from='client.44160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:05:49.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:49 smithi161 ceph-mon[26055]: from='client.? 172.21.15.116:0/606993608' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:05:49.911 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:49 smithi116 ceph-mon[100606]: from='client.64114 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:05:49.912 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:49 smithi116 ceph-mon[100606]: pgmap v42: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:49.912 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:49 smithi116 ceph-mon[100606]: from='client.44160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:05:49.912 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:49 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/606993608' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:05:49.912 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:49 smithi116 ceph-mon[89502]: from='client.64114 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:05:49.912 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:49 smithi116 ceph-mon[89502]: pgmap v42: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:49.912 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:49 smithi116 ceph-mon[89502]: from='client.44160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:05:49.912 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:49 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/606993608' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:05:50.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:49 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-b[26039]: 2024-06-01T02:05:49.911+0000 7f8cb0ae1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-01T02:05:50.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:49 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-b[26039]: 2024-06-01T02:05:49.911+0000 7f8cb0ae1700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-06-01T02:05:50.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:49 smithi161 podman[84233]: 2024-06-01 02:05:49.963238471 +0000 UTC m=+0.183122075 container died 9011986aa6e3b94168856b319ec79eb6ca37fd50c21f6d813d4b4ada0047ac78 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-b, build-date=2022-05-03T08:36:31.336870, io.buildah.version=1.19.8, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, name=centos-stream, com.redhat.component=centos-stream-container, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, version=8, architecture=x86_64, ceph=True, GIT_BRANCH=HEAD, RELEASE=HEAD, io.openshift.expose-services=, vendor=Red Hat, Inc., maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, io.openshift.tags=base centos centos-stream, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, GIT_CLEAN=True, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, CEPH_POINT_RELEASE=-17.2.0, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., distribution-scope=public, release=754, vcs-type=git, io.k8s.display-name=CentOS Stream 8) 2024-06-01T02:05:50.195 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:05:49 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: [01/Jun/2024:02:05:49] ENGINE Bus STOPPING 2024-06-01T02:05:50.195 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:05:49 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: [01/Jun/2024:02:05:49] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-06-01T02:05:50.195 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:05:49 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: [01/Jun/2024:02:05:49] ENGINE Bus STOPPED 2024-06-01T02:05:50.195 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:05:49 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: [01/Jun/2024:02:05:49] ENGINE Bus STARTING 2024-06-01T02:05:50.195 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:05:50 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: [01/Jun/2024:02:05:50] ENGINE Serving on http://:::9283 2024-06-01T02:05:50.195 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:05:50 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: [01/Jun/2024:02:05:50] ENGINE Bus STARTED 2024-06-01T02:05:50.616 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:50 smithi161 podman[84233]: 2024-06-01 02:05:50.319731223 +0000 UTC m=+0.539614828 container cleanup 9011986aa6e3b94168856b319ec79eb6ca37fd50c21f6d813d4b4ada0047ac78 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-b, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, vendor=Red Hat, Inc., CEPH_POINT_RELEASE=-17.2.0, ceph=True, maintainer=Guillaume Abrioux , io.openshift.tags=base centos centos-stream, io.buildah.version=1.19.8, RELEASE=HEAD, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, com.redhat.component=centos-stream-container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, GIT_CLEAN=True, vcs-type=git, io.openshift.expose-services=, release=754, GIT_REPO=https://github.com/ceph/ceph-container.git, version=8, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, build-date=2022-05-03T08:36:31.336870, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, io.k8s.display-name=CentOS Stream 8, distribution-scope=public, name=centos-stream, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, GIT_BRANCH=HEAD, architecture=x86_64) 2024-06-01T02:05:50.616 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:50 smithi161 bash[84233]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-b 2024-06-01T02:05:50.897 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:50 smithi161 podman[84258]: 2024-06-01 02:05:50.621150286 +0000 UTC m=+0.655884678 container remove 9011986aa6e3b94168856b319ec79eb6ca37fd50c21f6d813d4b4ada0047ac78 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-b, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, io.buildah.version=1.19.8, vendor=Red Hat, Inc., GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, architecture=x86_64, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, vcs-type=git, version=8, distribution-scope=public, io.openshift.expose-services=, io.openshift.tags=base centos centos-stream, maintainer=Guillaume Abrioux , RELEASE=HEAD, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, name=centos-stream, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, build-date=2022-05-03T08:36:31.336870, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, CEPH_POINT_RELEASE=-17.2.0, io.k8s.display-name=CentOS Stream 8, release=754, GIT_CLEAN=True, ceph=True, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, com.redhat.component=centos-stream-container, com.redhat.license_terms=https://centos.org/legal/licensing-policy/) 2024-06-01T02:05:50.897 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:50 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mon.b.service: Deactivated successfully. 2024-06-01T02:05:50.897 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:50 smithi161 systemd[1]: Stopped Ceph mon.b for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:05:50.897 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:50 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mon.b.service: Consumed 15.048s CPU time. 2024-06-01T02:05:51.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 systemd[1]: Starting Ceph mon.b for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:05:51.477 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 podman[84386]: 2024-06-01 02:05:51.11946691 +0000 UTC m=+0.017963714 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:05:51.477 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 podman[84386]: 2024-06-01 02:05:51.23925375 +0000 UTC m=+0.137750534 container create aaa08c03cfc87c81be34af9daf0b4aeefcb8473cbe3ebb93fb68eebda6ff9ca4 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-b, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-06-01T02:05:51.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 podman[84386]: 2024-06-01 02:05:51.48180061 +0000 UTC m=+0.380297401 container init aaa08c03cfc87c81be34af9daf0b4aeefcb8473cbe3ebb93fb68eebda6ff9ca4 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-01T02:05:51.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 podman[84386]: 2024-06-01 02:05:51.485214683 +0000 UTC m=+0.383711474 container start aaa08c03cfc87c81be34af9daf0b4aeefcb8473cbe3ebb93fb68eebda6ff9ca4 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc) 2024-06-01T02:05:51.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: set uid:gid to 167:167 (ceph:ceph) 2024-06-01T02:05:51.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc), process ceph-mon, pid 2 2024-06-01T02:05:51.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: pidfile_write: ignore empty --pid-file 2024-06-01T02:05:51.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: load: jerasure load: lrc 2024-06-01T02:05:51.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: RocksDB version: 7.9.2 2024-06-01T02:05:51.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Git sha 0 2024-06-01T02:05:51.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Compile date 2024-05-30 15:42:47 2024-06-01T02:05:51.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: DB SUMMARY 2024-06-01T02:05:51.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: DB Session ID: ZLRK9P6CTCNBKW70JQ7Q 2024-06-01T02:05:51.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: CURRENT file: CURRENT 2024-06-01T02:05:51.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: IDENTITY file: IDENTITY 2024-06-01T02:05:51.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: MANIFEST file: MANIFEST-000009 size: 2238 Bytes 2024-06-01T02:05:51.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 1, files: 000045.sst 2024-06-01T02:05:51.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000043.log size: 696518 ; 2024-06-01T02:05:51.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.error_if_exists: 0 2024-06-01T02:05:51.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.create_if_missing: 0 2024-06-01T02:05:51.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.paranoid_checks: 1 2024-06-01T02:05:51.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-01T02:05:51.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-01T02:05:51.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-01T02:05:51.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.env: 0x557dcbb05c60 2024-06-01T02:05:51.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.fs: PosixFileSystem 2024-06-01T02:05:51.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.info_log: 0x557dcc695b20 2024-06-01T02:05:51.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.max_file_opening_threads: 16 2024-06-01T02:05:51.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.statistics: (nil) 2024-06-01T02:05:51.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.use_fsync: 0 2024-06-01T02:05:51.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.max_log_file_size: 0 2024-06-01T02:05:51.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-01T02:05:51.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-01T02:05:51.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.keep_log_file_num: 1000 2024-06-01T02:05:51.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.recycle_log_file_num: 0 2024-06-01T02:05:51.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.allow_fallocate: 1 2024-06-01T02:05:51.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.allow_mmap_reads: 0 2024-06-01T02:05:51.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.allow_mmap_writes: 0 2024-06-01T02:05:51.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.use_direct_reads: 0 2024-06-01T02:05:51.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-01T02:05:51.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.create_missing_column_families: 0 2024-06-01T02:05:51.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.db_log_dir: 2024-06-01T02:05:51.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.wal_dir: 2024-06-01T02:05:51.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-01T02:05:51.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-01T02:05:51.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-01T02:05:51.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-01T02:05:51.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-01T02:05:51.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-01T02:05:51.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.advise_random_on_open: 1 2024-06-01T02:05:51.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.db_write_buffer_size: 0 2024-06-01T02:05:51.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.write_buffer_manager: 0x557dcc666820 2024-06-01T02:05:51.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-01T02:05:51.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-01T02:05:51.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-01T02:05:51.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.rate_limiter: (nil) 2024-06-01T02:05:51.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-01T02:05:51.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.wal_recovery_mode: 2 2024-06-01T02:05:51.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.enable_thread_tracking: 0 2024-06-01T02:05:51.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.enable_pipelined_write: 0 2024-06-01T02:05:51.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.unordered_write: 0 2024-06-01T02:05:51.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-01T02:05:51.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-01T02:05:51.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-01T02:05:51.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-01T02:05:51.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.row_cache: None 2024-06-01T02:05:51.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.wal_filter: None 2024-06-01T02:05:51.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-01T02:05:51.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.allow_ingest_behind: 0 2024-06-01T02:05:51.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.two_write_queues: 0 2024-06-01T02:05:51.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.manual_wal_flush: 0 2024-06-01T02:05:51.950 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.wal_compression: 0 2024-06-01T02:05:51.951 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.atomic_flush: 0 2024-06-01T02:05:51.951 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-01T02:05:51.951 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-01T02:05:51.951 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-01T02:05:51.951 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.log_readahead_size: 0 2024-06-01T02:05:51.951 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-01T02:05:51.951 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.best_efforts_recovery: 0 2024-06-01T02:05:51.951 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-01T02:05:51.951 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-01T02:05:51.951 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.allow_data_in_errors: 0 2024-06-01T02:05:51.951 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.db_host_id: __hostname__ 2024-06-01T02:05:51.951 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-01T02:05:51.951 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.max_background_jobs: 2 2024-06-01T02:05:51.951 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.max_background_compactions: -1 2024-06-01T02:05:51.951 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.max_subcompactions: 1 2024-06-01T02:05:51.952 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-01T02:05:51.952 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-01T02:05:51.952 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-01T02:05:51.952 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.max_total_wal_size: 0 2024-06-01T02:05:51.952 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-01T02:05:51.952 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-01T02:05:51.952 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-01T02:05:51.952 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-01T02:05:51.952 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.max_open_files: -1 2024-06-01T02:05:51.953 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.bytes_per_sync: 0 2024-06-01T02:05:51.953 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-01T02:05:51.953 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-01T02:05:51.953 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.compaction_readahead_size: 0 2024-06-01T02:05:51.953 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.max_background_flushes: -1 2024-06-01T02:05:51.953 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Compression algorithms supported: 2024-06-01T02:05:51.953 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-01T02:05:51.953 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: kZSTD supported: 0 2024-06-01T02:05:51.953 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: kXpressCompression supported: 0 2024-06-01T02:05:51.953 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: kLZ4HCCompression supported: 1 2024-06-01T02:05:51.953 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: kZlibCompression supported: 1 2024-06-01T02:05:51.953 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: kSnappyCompression supported: 1 2024-06-01T02:05:51.953 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: kLZ4Compression supported: 1 2024-06-01T02:05:51.953 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: kBZip2Compression supported: 0 2024-06-01T02:05:51.954 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-01T02:05:51.954 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-01T02:05:51.954 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000009 2024-06-01T02:05:51.954 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-01T02:05:51.954 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-01T02:05:51.954 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.merge_operator: 2024-06-01T02:05:51.954 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.compaction_filter: None 2024-06-01T02:05:51.954 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.compaction_filter_factory: None 2024-06-01T02:05:51.954 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.sst_partitioner_factory: None 2024-06-01T02:05:51.954 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-01T02:05:51.954 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-01T02:05:51.954 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557dcc695c40) 2024-06-01T02:05:51.954 INFO:journalctl@ceph.mon.b.smithi161.stdout: cache_index_and_filter_blocks: 1 2024-06-01T02:05:51.955 INFO:journalctl@ceph.mon.b.smithi161.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-01T02:05:51.955 INFO:journalctl@ceph.mon.b.smithi161.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-01T02:05:51.955 INFO:journalctl@ceph.mon.b.smithi161.stdout: pin_top_level_index_and_filter: 1 2024-06-01T02:05:51.955 INFO:journalctl@ceph.mon.b.smithi161.stdout: index_type: 0 2024-06-01T02:05:51.955 INFO:journalctl@ceph.mon.b.smithi161.stdout: data_block_index_type: 0 2024-06-01T02:05:51.955 INFO:journalctl@ceph.mon.b.smithi161.stdout: index_shortening: 1 2024-06-01T02:05:51.955 INFO:journalctl@ceph.mon.b.smithi161.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-01T02:05:51.955 INFO:journalctl@ceph.mon.b.smithi161.stdout: checksum: 4 2024-06-01T02:05:51.955 INFO:journalctl@ceph.mon.b.smithi161.stdout: no_block_cache: 0 2024-06-01T02:05:51.955 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_cache: 0x557dcc697090 2024-06-01T02:05:51.955 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_cache_name: BinnedLRUCache 2024-06-01T02:05:51.955 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_cache_options: 2024-06-01T02:05:51.955 INFO:journalctl@ceph.mon.b.smithi161.stdout: capacity : 536870912 2024-06-01T02:05:51.955 INFO:journalctl@ceph.mon.b.smithi161.stdout: num_shard_bits : 4 2024-06-01T02:05:51.956 INFO:journalctl@ceph.mon.b.smithi161.stdout: strict_capacity_limit : 0 2024-06-01T02:05:51.956 INFO:journalctl@ceph.mon.b.smithi161.stdout: high_pri_pool_ratio: 0.000 2024-06-01T02:05:51.956 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_cache_compressed: (nil) 2024-06-01T02:05:51.956 INFO:journalctl@ceph.mon.b.smithi161.stdout: persistent_cache: (nil) 2024-06-01T02:05:51.956 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_size: 4096 2024-06-01T02:05:51.956 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_size_deviation: 10 2024-06-01T02:05:51.956 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_restart_interval: 16 2024-06-01T02:05:51.956 INFO:journalctl@ceph.mon.b.smithi161.stdout: index_block_restart_interval: 1 2024-06-01T02:05:51.956 INFO:journalctl@ceph.mon.b.smithi161.stdout: metadata_block_size: 4096 2024-06-01T02:05:51.956 INFO:journalctl@ceph.mon.b.smithi161.stdout: partition_filters: 0 2024-06-01T02:05:51.956 INFO:journalctl@ceph.mon.b.smithi161.stdout: use_delta_encoding: 1 2024-06-01T02:05:51.956 INFO:journalctl@ceph.mon.b.smithi161.stdout: filter_policy: bloomfilter 2024-06-01T02:05:51.956 INFO:journalctl@ceph.mon.b.smithi161.stdout: whole_key_filtering: 1 2024-06-01T02:05:51.956 INFO:journalctl@ceph.mon.b.smithi161.stdout: verify_compression: 0 2024-06-01T02:05:51.957 INFO:journalctl@ceph.mon.b.smithi161.stdout: read_amp_bytes_per_bit: 0 2024-06-01T02:05:51.957 INFO:journalctl@ceph.mon.b.smithi161.stdout: format_version: 5 2024-06-01T02:05:51.957 INFO:journalctl@ceph.mon.b.smithi161.stdout: enable_index_compression: 1 2024-06-01T02:05:51.957 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_align: 0 2024-06-01T02:05:51.957 INFO:journalctl@ceph.mon.b.smithi161.stdout: max_auto_readahead_size: 262144 2024-06-01T02:05:51.957 INFO:journalctl@ceph.mon.b.smithi161.stdout: prepopulate_block_cache: 0 2024-06-01T02:05:51.957 INFO:journalctl@ceph.mon.b.smithi161.stdout: initial_auto_readahead_size: 8192 2024-06-01T02:05:51.957 INFO:journalctl@ceph.mon.b.smithi161.stdout: num_file_reads_for_auto_readahead: 2 2024-06-01T02:05:51.957 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.write_buffer_size: 33554432 2024-06-01T02:05:51.957 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.max_write_buffer_number: 2 2024-06-01T02:05:51.957 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.compression: NoCompression 2024-06-01T02:05:51.957 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.bottommost_compression: Disabled 2024-06-01T02:05:51.957 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.prefix_extractor: nullptr 2024-06-01T02:05:51.957 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-01T02:05:51.958 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.num_levels: 7 2024-06-01T02:05:51.958 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-01T02:05:51.958 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-01T02:05:51.958 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-01T02:05:51.958 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-01T02:05:51.958 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-01T02:05:51.958 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-01T02:05:51.958 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-01T02:05:51.958 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-01T02:05:51.958 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-01T02:05:51.958 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-01T02:05:51.958 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-01T02:05:51.958 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-01T02:05:51.958 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-01T02:05:51.959 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.compression_opts.level: 32767 2024-06-01T02:05:51.959 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.compression_opts.strategy: 0 2024-06-01T02:05:51.959 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-01T02:05:51.959 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-01T02:05:51.959 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-01T02:05:51.959 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-01T02:05:51.959 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.compression_opts.enabled: false 2024-06-01T02:05:51.959 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-01T02:05:51.959 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-01T02:05:51.959 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-01T02:05:51.959 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-01T02:05:51.959 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.target_file_size_base: 67108864 2024-06-01T02:05:51.959 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-01T02:05:51.959 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-01T02:05:51.960 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-01T02:05:51.960 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-01T02:05:51.960 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-01T02:05:51.960 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-01T02:05:51.960 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-01T02:05:51.960 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-01T02:05:51.960 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-01T02:05:51.960 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-01T02:05:51.960 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-01T02:05:51.960 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-01T02:05:51.960 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-01T02:05:51.960 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-01T02:05:51.960 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.arena_block_size: 1048576 2024-06-01T02:05:51.960 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-01T02:05:51.961 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-01T02:05:51.961 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.disable_auto_compactions: 0 2024-06-01T02:05:51.961 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-01T02:05:51.961 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-01T02:05:51.961 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-01T02:05:51.961 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-01T02:05:51.961 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-01T02:05:51.961 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-01T02:05:51.961 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-01T02:05:51.961 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-01T02:05:51.961 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-01T02:05:51.961 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-01T02:05:51.961 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.table_properties_collectors: 2024-06-01T02:05:51.961 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.inplace_update_support: 0 2024-06-01T02:05:51.962 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-01T02:05:51.962 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-01T02:05:51.962 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-01T02:05:51.962 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-01T02:05:51.962 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.bloom_locality: 0 2024-06-01T02:05:51.962 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.max_successive_merges: 0 2024-06-01T02:05:51.962 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-01T02:05:51.962 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.paranoid_file_checks: 0 2024-06-01T02:05:51.962 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.force_consistency_checks: 1 2024-06-01T02:05:51.962 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.report_bg_io_stats: 0 2024-06-01T02:05:51.962 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.ttl: 2592000 2024-06-01T02:05:51.962 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-01T02:05:51.962 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-01T02:05:51.962 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-01T02:05:51.963 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.enable_blob_files: false 2024-06-01T02:05:51.963 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.min_blob_size: 0 2024-06-01T02:05:51.963 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.blob_file_size: 268435456 2024-06-01T02:05:51.963 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-01T02:05:51.963 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-01T02:05:51.963 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-01T02:05:51.963 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-01T02:05:51.963 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-01T02:05:51.963 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.blob_file_starting_level: 0 2024-06-01T02:05:51.963 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-01T02:05:51.963 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 45.sst 2024-06-01T02:05:51.963 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-06-01T02:05:51.963 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 47, last_sequence is 29465, log_number is 43,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-06-01T02:05:51.963 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 43 2024-06-01T02:05:51.964 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 52f80fb8-51c7-4fb8-bba9-caaacdea0c61 2024-06-01T02:05:51.964 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717207551525444, "job": 1, "event": "recovery_started", "wal_files": [43]} 2024-06-01T02:05:51.964 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #43 mode 2 2024-06-01T02:05:51.964 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717207551528004, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 48, "file_size": 439915, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 29470, "largest_seqno": 30030, "table_properties": {"data_size": 437020, "index_size": 1049, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 837, "raw_key_size": 8183, "raw_average_key_size": 25, "raw_value_size": 431319, "raw_average_value_size": 1323, "num_data_blocks": 43, "num_entries": 326, "num_filter_entries": 326, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1717207551, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "52f80fb8-51c7-4fb8-bba9-caaacdea0c61", "db_session_id": "ZLRK9P6CTCNBKW70JQ7Q", "orig_file_number": 48, "seqno_to_time_mapping": "N/A"}} 2024-06-01T02:05:51.964 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717207551528088, "job": 1, "event": "recovery_finished"} 2024-06-01T02:05:51.964 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: [db/version_set.cc:5047] Creating manifest 50 2024-06-01T02:05:51.964 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-06-01T02:05:51.964 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000043.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-01T02:05:51.964 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557dcc786000 2024-06-01T02:05:51.964 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: DB pointer 0x557dcc770000 2024-06-01T02:05:51.964 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-01T02:05:51.964 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-01T02:05:51.964 INFO:journalctl@ceph.mon.b.smithi161.stdout: ** DB Stats ** 2024-06-01T02:05:51.965 INFO:journalctl@ceph.mon.b.smithi161.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-01T02:05:51.965 INFO:journalctl@ceph.mon.b.smithi161.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-01T02:05:51.965 INFO:journalctl@ceph.mon.b.smithi161.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-01T02:05:51.965 INFO:journalctl@ceph.mon.b.smithi161.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-01T02:05:51.965 INFO:journalctl@ceph.mon.b.smithi161.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-01T02:05:51.965 INFO:journalctl@ceph.mon.b.smithi161.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-01T02:05:51.965 INFO:journalctl@ceph.mon.b.smithi161.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-01T02:05:51.965 INFO:journalctl@ceph.mon.b.smithi161.stdout: 2024-06-01T02:05:51.965 INFO:journalctl@ceph.mon.b.smithi161.stdout: ** Compaction Stats [default] ** 2024-06-01T02:05:51.965 INFO:journalctl@ceph.mon.b.smithi161.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-01T02:05:51.965 INFO:journalctl@ceph.mon.b.smithi161.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-01T02:05:51.965 INFO:journalctl@ceph.mon.b.smithi161.stdout: L0 1/0 429.60 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 306.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-01T02:05:51.965 INFO:journalctl@ceph.mon.b.smithi161.stdout: L6 1/0 11.15 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-06-01T02:05:51.965 INFO:journalctl@ceph.mon.b.smithi161.stdout: Sum 2/0 11.57 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 306.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-01T02:05:51.965 INFO:journalctl@ceph.mon.b.smithi161.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 306.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-01T02:05:51.966 INFO:journalctl@ceph.mon.b.smithi161.stdout: 2024-06-01T02:05:51.966 INFO:journalctl@ceph.mon.b.smithi161.stdout: ** Compaction Stats [default] ** 2024-06-01T02:05:51.966 INFO:journalctl@ceph.mon.b.smithi161.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-01T02:05:51.966 INFO:journalctl@ceph.mon.b.smithi161.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-01T02:05:51.966 INFO:journalctl@ceph.mon.b.smithi161.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 306.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-01T02:05:51.966 INFO:journalctl@ceph.mon.b.smithi161.stdout: 2024-06-01T02:05:51.966 INFO:journalctl@ceph.mon.b.smithi161.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-01T02:05:51.966 INFO:journalctl@ceph.mon.b.smithi161.stdout: 2024-06-01T02:05:51.966 INFO:journalctl@ceph.mon.b.smithi161.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-01T02:05:51.966 INFO:journalctl@ceph.mon.b.smithi161.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-01T02:05:51.966 INFO:journalctl@ceph.mon.b.smithi161.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-01T02:05:51.966 INFO:journalctl@ceph.mon.b.smithi161.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-01T02:05:51.966 INFO:journalctl@ceph.mon.b.smithi161.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-01T02:05:51.966 INFO:journalctl@ceph.mon.b.smithi161.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-01T02:05:51.967 INFO:journalctl@ceph.mon.b.smithi161.stdout: Cumulative compaction: 0.00 GB write, 27.14 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-01T02:05:51.967 INFO:journalctl@ceph.mon.b.smithi161.stdout: Interval compaction: 0.00 GB write, 27.14 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-01T02:05:51.967 INFO:journalctl@ceph.mon.b.smithi161.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-01T02:05:51.967 INFO:journalctl@ceph.mon.b.smithi161.stdout: Block cache BinnedLRUCache@0x557dcc697090#2 capacity: 512.00 MB usage: 2.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-06-01T02:05:51.967 INFO:journalctl@ceph.mon.b.smithi161.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.91 KB,0.000172853%) IndexBlock(1,1.20 KB,0.000229478%) Misc(1,0.00 KB,0%) 2024-06-01T02:05:51.967 INFO:journalctl@ceph.mon.b.smithi161.stdout: 2024-06-01T02:05:51.967 INFO:journalctl@ceph.mon.b.smithi161.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-01T02:05:51.967 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: starting mon.b rank 2 at public addrs [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] at bind addrs [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T02:05:51.967 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: mon.b@-1(???) e3 preinit fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T02:05:51.967 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: mon.b@-1(???).mds e1 new map 2024-06-01T02:05:51.967 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: mon.b@-1(???).mds e1 print_map 2024-06-01T02:05:51.967 INFO:journalctl@ceph.mon.b.smithi161.stdout: e1 2024-06-01T02:05:51.967 INFO:journalctl@ceph.mon.b.smithi161.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-01T02:05:51.967 INFO:journalctl@ceph.mon.b.smithi161.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-01T02:05:51.968 INFO:journalctl@ceph.mon.b.smithi161.stdout: legacy client fscid: -1 2024-06-01T02:05:51.968 INFO:journalctl@ceph.mon.b.smithi161.stdout: 2024-06-01T02:05:51.968 INFO:journalctl@ceph.mon.b.smithi161.stdout: No filesystems configured 2024-06-01T02:05:51.968 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: mon.b@-1(???).osd e114 crush map has features 3314933000854323200, adjusting msgr requires 2024-06-01T02:05:51.968 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: mon.b@-1(???).osd e114 crush map has features 432629239337189376, adjusting msgr requires 2024-06-01T02:05:51.968 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: mon.b@-1(???).osd e114 crush map has features 432629239337189376, adjusting msgr requires 2024-06-01T02:05:51.968 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: mon.b@-1(???).osd e114 crush map has features 432629239337189376, adjusting msgr requires 2024-06-01T02:05:51.968 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: mon.b@-1(???).paxosservice(auth 1..28) refresh upgraded, format 0 -> 3 2024-06-01T02:05:51.968 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 bash[84386]: aaa08c03cfc87c81be34af9daf0b4aeefcb8473cbe3ebb93fb68eebda6ff9ca4 2024-06-01T02:05:51.968 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 systemd[1]: Started Ceph mon.b for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:05:51.968 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T02:05:51.968 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T02:05:51.968 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T02:05:52.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T02:05:52.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T02:05:52.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T02:05:52.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T02:05:52.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T02:05:52.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T02:05:52.932 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: fsmap 2024-06-01T02:05:52.932 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: osdmap e114: 8 total, 8 up, 8 in 2024-06-01T02:05:52.932 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: mgrmap e44: x(active, since 80s), standbys: y 2024-06-01T02:05:52.932 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: overall HEALTH_OK 2024-06-01T02:05:52.932 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: mon.b calling monitor election 2024-06-01T02:05:52.932 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: mon.c calling monitor election 2024-06-01T02:05:52.932 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: mon.a calling monitor election 2024-06-01T02:05:52.933 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T02:05:52.933 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T02:05:52.933 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T02:05:52.933 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-01T02:05:52.933 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: monmap epoch 4 2024-06-01T02:05:52.933 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T02:05:52.933 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: last_changed 2024-06-01T02:05:51.705570+0000 2024-06-01T02:05:52.933 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: created 2024-06-01T01:50:20.260187+0000 2024-06-01T02:05:52.933 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: min_mon_release 19 (squid) 2024-06-01T02:05:52.933 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: election_strategy: 1 2024-06-01T02:05:52.933 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: 0: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.a 2024-06-01T02:05:52.933 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: 1: [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] mon.c 2024-06-01T02:05:52.933 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: 2: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-06-01T02:05:52.933 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: fsmap 2024-06-01T02:05:52.934 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: osdmap e114: 8 total, 8 up, 8 in 2024-06-01T02:05:52.934 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: mgrmap e44: x(active, since 80s), standbys: y 2024-06-01T02:05:52.934 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: overall HEALTH_OK 2024-06-01T02:05:52.934 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:52.934 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:52.934 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:52.934 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:52.934 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:52.934 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:52.934 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:05:52.934 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: Reconfiguring mon.a (monmap changed)... 2024-06-01T02:05:52.934 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T02:05:52.934 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T02:05:52.935 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:52.935 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[100606]: Reconfiguring daemon mon.a on smithi116 2024-06-01T02:05:52.935 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: fsmap 2024-06-01T02:05:52.935 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: osdmap e114: 8 total, 8 up, 8 in 2024-06-01T02:05:52.935 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: mgrmap e44: x(active, since 80s), standbys: y 2024-06-01T02:05:52.935 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: overall HEALTH_OK 2024-06-01T02:05:52.935 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: mon.b calling monitor election 2024-06-01T02:05:52.935 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: mon.c calling monitor election 2024-06-01T02:05:52.935 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: mon.a calling monitor election 2024-06-01T02:05:52.935 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T02:05:52.936 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T02:05:52.936 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T02:05:52.936 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-01T02:05:52.936 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: monmap epoch 4 2024-06-01T02:05:52.936 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T02:05:52.936 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: last_changed 2024-06-01T02:05:51.705570+0000 2024-06-01T02:05:52.936 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: created 2024-06-01T01:50:20.260187+0000 2024-06-01T02:05:52.936 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: min_mon_release 19 (squid) 2024-06-01T02:05:52.936 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: election_strategy: 1 2024-06-01T02:05:52.936 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: 0: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.a 2024-06-01T02:05:52.936 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: 1: [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] mon.c 2024-06-01T02:05:52.936 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: 2: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-06-01T02:05:52.936 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: fsmap 2024-06-01T02:05:52.937 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: osdmap e114: 8 total, 8 up, 8 in 2024-06-01T02:05:52.937 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: mgrmap e44: x(active, since 80s), standbys: y 2024-06-01T02:05:52.937 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: overall HEALTH_OK 2024-06-01T02:05:52.937 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:52.937 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:52.937 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:52.937 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:52.937 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:52.937 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:52.937 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:05:52.937 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: Reconfiguring mon.a (monmap changed)... 2024-06-01T02:05:52.937 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T02:05:52.937 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T02:05:52.937 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:52.938 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:52 smithi116 ceph-mon[89502]: Reconfiguring daemon mon.a on smithi116 2024-06-01T02:05:53.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: fsmap 2024-06-01T02:05:53.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: osdmap e114: 8 total, 8 up, 8 in 2024-06-01T02:05:53.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: mgrmap e44: x(active, since 80s), standbys: y 2024-06-01T02:05:53.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: overall HEALTH_OK 2024-06-01T02:05:53.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: mon.b calling monitor election 2024-06-01T02:05:53.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: mon.c calling monitor election 2024-06-01T02:05:53.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: mon.a calling monitor election 2024-06-01T02:05:53.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-01T02:05:53.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-01T02:05:53.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-01T02:05:53.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-01T02:05:53.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: monmap epoch 4 2024-06-01T02:05:53.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 2024-06-01T02:05:53.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: last_changed 2024-06-01T02:05:51.705570+0000 2024-06-01T02:05:53.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: created 2024-06-01T01:50:20.260187+0000 2024-06-01T02:05:53.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: min_mon_release 19 (squid) 2024-06-01T02:05:53.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: election_strategy: 1 2024-06-01T02:05:53.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: 0: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.a 2024-06-01T02:05:53.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: 1: [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] mon.c 2024-06-01T02:05:53.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: 2: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-06-01T02:05:53.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: fsmap 2024-06-01T02:05:53.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: osdmap e114: 8 total, 8 up, 8 in 2024-06-01T02:05:53.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: mgrmap e44: x(active, since 80s), standbys: y 2024-06-01T02:05:53.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: overall HEALTH_OK 2024-06-01T02:05:53.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:53.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:53.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:53.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:05:53.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:53.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:05:53.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:05:53.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: Reconfiguring mon.a (monmap changed)... 2024-06-01T02:05:53.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T02:05:53.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T02:05:53.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:53.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:52 smithi161 ceph-mon[84406]: Reconfiguring daemon mon.a on smithi116 2024-06-01T02:05:53.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:53 smithi161 ceph-mon[84406]: pgmap v44: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:53.944 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:05:53 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:05:53.505Z caller=manager.go:609 level=warn component="rule manager" group=nodes msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on(instance) group_left(nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi116\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:05:54.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:53 smithi116 ceph-mon[89502]: pgmap v44: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:54.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:53 smithi116 ceph-mon[100606]: pgmap v44: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:55.257 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:55.257 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:55.257 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:54 smithi161 ceph-mon[84406]: Reconfiguring mgr.y (monmap changed)... 2024-06-01T02:05:55.257 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:05:55.257 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-01T02:05:55.257 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:55.258 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:54 smithi161 ceph-mon[84406]: Reconfiguring daemon mgr.y on smithi116 2024-06-01T02:05:55.258 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:54 smithi161 ceph-mon[84406]: pgmap v45: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:55.271 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:55.271 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:55.271 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:54 smithi116 ceph-mon[100606]: Reconfiguring mgr.y (monmap changed)... 2024-06-01T02:05:55.271 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:05:55.271 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-01T02:05:55.271 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:55.271 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:54 smithi116 ceph-mon[100606]: Reconfiguring daemon mgr.y on smithi116 2024-06-01T02:05:55.271 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:54 smithi116 ceph-mon[100606]: pgmap v45: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:55.271 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:55.271 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:55.272 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:54 smithi116 ceph-mon[89502]: Reconfiguring mgr.y (monmap changed)... 2024-06-01T02:05:55.272 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:05:55.272 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-01T02:05:55.272 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:55.272 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:54 smithi116 ceph-mon[89502]: Reconfiguring daemon mgr.y on smithi116 2024-06-01T02:05:55.272 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:54 smithi116 ceph-mon[89502]: pgmap v45: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:56.694 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:05:56 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:05:56.538+0000 7fcedaabb700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-06-01T02:05:56.966 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:56.966 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:56.966 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:56 smithi161 ceph-mon[84406]: Reconfiguring mon.c (monmap changed)... 2024-06-01T02:05:56.966 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T02:05:56.966 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T02:05:56.967 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:56.967 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:56 smithi161 ceph-mon[84406]: Reconfiguring daemon mon.c on smithi116 2024-06-01T02:05:56.967 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:05:56 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:05:56] "GET /metrics HTTP/1.1" 200 38261 "" "Prometheus/2.33.4" 2024-06-01T02:05:57.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:57.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:57.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:56 smithi116 ceph-mon[100606]: Reconfiguring mon.c (monmap changed)... 2024-06-01T02:05:57.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T02:05:57.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T02:05:57.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:57.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:56 smithi116 ceph-mon[100606]: Reconfiguring daemon mon.c on smithi116 2024-06-01T02:05:57.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:57.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:57.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:56 smithi116 ceph-mon[89502]: Reconfiguring mon.c (monmap changed)... 2024-06-01T02:05:57.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T02:05:57.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T02:05:57.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:57.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:56 smithi116 ceph-mon[89502]: Reconfiguring daemon mon.c on smithi116 2024-06-01T02:05:58.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:57 smithi116 ceph-mon[89502]: pgmap v46: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:58.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:58.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:58.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-01T02:05:58.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:58.010 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:57 smithi116 ceph-mon[100606]: pgmap v46: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:58.010 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:57 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:58.010 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:57 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:58.011 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:57 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-01T02:05:58.011 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:57 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:58.190 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:57 smithi161 ceph-mon[84406]: pgmap v46: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:05:58.191 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:57 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:58.191 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:57 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:58.191 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:57 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-01T02:05:58.191 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:57 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:05:58.694 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:05:58 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:05:58.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"172.21.15.161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"smithi161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:05:59.019 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:58 smithi116 ceph-mon[100606]: Reconfiguring osd.0 (monmap changed)... 2024-06-01T02:05:59.019 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:58 smithi116 ceph-mon[100606]: Reconfiguring daemon osd.0 on smithi116 2024-06-01T02:05:59.019 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:58 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:59.019 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:58 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:59.020 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:58 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:59.020 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:58 smithi116 ceph-mon[89502]: Reconfiguring osd.0 (monmap changed)... 2024-06-01T02:05:59.020 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:58 smithi116 ceph-mon[89502]: Reconfiguring daemon osd.0 on smithi116 2024-06-01T02:05:59.020 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:59.020 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:59.020 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:58 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:59.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:58 smithi161 ceph-mon[84406]: Reconfiguring osd.0 (monmap changed)... 2024-06-01T02:05:59.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:58 smithi161 ceph-mon[84406]: Reconfiguring daemon osd.0 on smithi116 2024-06-01T02:05:59.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:58 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:59.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:58 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:59.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:58 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:59.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:59 smithi161 ceph-mon[84406]: pgmap v47: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:05:59.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:59 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:59.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:59 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:05:59.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:59 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-01T02:05:59.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:05:59 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:00.006 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:59 smithi116 ceph-mon[100606]: pgmap v47: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:06:00.006 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:59 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:00.006 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:59 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:00.006 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:59 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-01T02:06:00.006 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:05:59 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:00.006 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:59 smithi116 ceph-mon[89502]: pgmap v47: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:06:00.006 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:00.006 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:00.006 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-01T02:06:00.007 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:05:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:01.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:00 smithi116 ceph-mon[89502]: Reconfiguring osd.1 (monmap changed)... 2024-06-01T02:06:01.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:00 smithi116 ceph-mon[89502]: Reconfiguring daemon osd.1 on smithi116 2024-06-01T02:06:01.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:01.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:01.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-01T02:06:01.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:01.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:00 smithi116 ceph-mon[100606]: Reconfiguring osd.1 (monmap changed)... 2024-06-01T02:06:01.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:00 smithi116 ceph-mon[100606]: Reconfiguring daemon osd.1 on smithi116 2024-06-01T02:06:01.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:00 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:01.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:00 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:01.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:00 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-01T02:06:01.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:00 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:01.070 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:00 smithi161 ceph-mon[84406]: Reconfiguring osd.1 (monmap changed)... 2024-06-01T02:06:01.071 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:00 smithi161 ceph-mon[84406]: Reconfiguring daemon osd.1 on smithi116 2024-06-01T02:06:01.071 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:00 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:01.071 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:00 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:01.071 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:00 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-01T02:06:01.071 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:00 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:01.976 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:01 smithi116 ceph-mon[89502]: Reconfiguring osd.2 (monmap changed)... 2024-06-01T02:06:01.976 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:01 smithi116 ceph-mon[89502]: Reconfiguring daemon osd.2 on smithi116 2024-06-01T02:06:01.976 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:01 smithi116 ceph-mon[89502]: pgmap v48: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:06:01.976 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:06:01.976 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:01.976 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:01.976 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:01.976 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:01 smithi116 ceph-mon[100606]: Reconfiguring osd.2 (monmap changed)... 2024-06-01T02:06:01.977 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:01 smithi116 ceph-mon[100606]: Reconfiguring daemon osd.2 on smithi116 2024-06-01T02:06:01.977 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:01 smithi116 ceph-mon[100606]: pgmap v48: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:06:01.977 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:01 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:06:01.977 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:01 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:01.977 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:01 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:01.977 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:01 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:02.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:01 smithi161 ceph-mon[84406]: Reconfiguring osd.2 (monmap changed)... 2024-06-01T02:06:02.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:01 smithi161 ceph-mon[84406]: Reconfiguring daemon osd.2 on smithi116 2024-06-01T02:06:02.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:01 smithi161 ceph-mon[84406]: pgmap v48: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:06:02.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:01 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:06:02.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:01 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:02.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:01 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:02.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:01 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:03.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:02 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:06:03.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:03.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:03.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-01T02:06:03.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:03.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:02 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:06:03.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:02 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:03.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:02 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:03.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:02 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-01T02:06:03.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:02 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:03.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:02 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:06:03.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:02 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:03.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:02 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:03.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:02 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-01T02:06:03.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:02 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:03.774 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:06:03 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:06:03.505Z caller=manager.go:609 level=warn component="rule manager" group=nodes msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on(instance) group_left(nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi116\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:06:04.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:03 smithi116 ceph-mon[89502]: Reconfiguring osd.3 (monmap changed)... 2024-06-01T02:06:04.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:03 smithi116 ceph-mon[89502]: Reconfiguring daemon osd.3 on smithi116 2024-06-01T02:06:04.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:03 smithi116 ceph-mon[89502]: pgmap v49: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:06:04.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:03 smithi116 ceph-mon[100606]: Reconfiguring osd.3 (monmap changed)... 2024-06-01T02:06:04.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:03 smithi116 ceph-mon[100606]: Reconfiguring daemon osd.3 on smithi116 2024-06-01T02:06:04.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:03 smithi116 ceph-mon[100606]: pgmap v49: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:06:04.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:03 smithi161 ceph-mon[84406]: Reconfiguring osd.3 (monmap changed)... 2024-06-01T02:06:04.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:03 smithi161 ceph-mon[84406]: Reconfiguring daemon osd.3 on smithi116 2024-06-01T02:06:04.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:03 smithi161 ceph-mon[84406]: pgmap v49: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:06:05.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:05.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:05.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:05 smithi116 ceph-mon[100606]: Reconfiguring rgw.foo.smithi116.vzjrxj (monmap changed)... 2024-06-01T02:06:05.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi116.vzjrxj", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T02:06:05.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:05.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:05 smithi116 ceph-mon[100606]: Reconfiguring daemon rgw.foo.smithi116.vzjrxj on smithi116 2024-06-01T02:06:05.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:05 smithi116 ceph-mon[100606]: pgmap v50: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:06:05.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:05.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:05.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:05 smithi116 ceph-mon[89502]: Reconfiguring rgw.foo.smithi116.vzjrxj (monmap changed)... 2024-06-01T02:06:05.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi116.vzjrxj", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T02:06:05.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:05.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:05 smithi116 ceph-mon[89502]: Reconfiguring daemon rgw.foo.smithi116.vzjrxj on smithi116 2024-06-01T02:06:05.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:05 smithi116 ceph-mon[89502]: pgmap v50: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:06:05.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:05.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:05.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:05 smithi161 ceph-mon[84406]: Reconfiguring rgw.foo.smithi116.vzjrxj (monmap changed)... 2024-06-01T02:06:05.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi116.vzjrxj", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T02:06:05.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:05.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:05 smithi161 ceph-mon[84406]: Reconfiguring daemon rgw.foo.smithi116.vzjrxj on smithi116 2024-06-01T02:06:05.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:05 smithi161 ceph-mon[84406]: pgmap v50: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:06:07.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:07.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:07.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:06 smithi116 ceph-mon[89502]: Reconfiguring rgw.smpl.smithi116.hokbmf (monmap changed)... 2024-06-01T02:06:07.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi116.hokbmf", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T02:06:07.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:07.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:06 smithi116 ceph-mon[89502]: Reconfiguring daemon rgw.smpl.smithi116.hokbmf on smithi116 2024-06-01T02:06:07.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:07.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:07.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:06 smithi116 ceph-mon[100606]: Reconfiguring rgw.smpl.smithi116.hokbmf (monmap changed)... 2024-06-01T02:06:07.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi116.hokbmf", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T02:06:07.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:07.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:06 smithi116 ceph-mon[100606]: Reconfiguring daemon rgw.smpl.smithi116.hokbmf on smithi116 2024-06-01T02:06:07.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:06:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:06:06] "GET /metrics HTTP/1.1" 200 38276 "" "Prometheus/2.33.4" 2024-06-01T02:06:07.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:07.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:07.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:06 smithi161 ceph-mon[84406]: Reconfiguring rgw.smpl.smithi116.hokbmf (monmap changed)... 2024-06-01T02:06:07.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi116.hokbmf", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T02:06:07.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:07.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:06 smithi161 ceph-mon[84406]: Reconfiguring daemon rgw.smpl.smithi116.hokbmf on smithi116 2024-06-01T02:06:08.062 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:07 smithi161 ceph-mon[84406]: pgmap v51: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:06:08.062 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:08.062 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:08.063 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T02:06:08.063 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T02:06:08.063 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:08.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:07 smithi116 ceph-mon[100606]: pgmap v51: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:06:08.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:08.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:08.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T02:06:08.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T02:06:08.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:08.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:07 smithi116 ceph-mon[89502]: pgmap v51: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:06:08.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:08.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:08.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-01T02:06:08.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-01T02:06:08.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:08.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:08 smithi161 ceph-mon[84406]: Reconfiguring mon.b (monmap changed)... 2024-06-01T02:06:08.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:08 smithi161 ceph-mon[84406]: Reconfiguring daemon mon.b on smithi161 2024-06-01T02:06:08.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:08.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:08.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:08.946 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:06:08 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:06:08.660Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"172.21.15.161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"smithi161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:06:09.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:08 smithi116 ceph-mon[89502]: Reconfiguring mon.b (monmap changed)... 2024-06-01T02:06:09.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:08 smithi116 ceph-mon[89502]: Reconfiguring daemon mon.b on smithi161 2024-06-01T02:06:09.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:09.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:09.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:09.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:08 smithi116 ceph-mon[100606]: Reconfiguring mon.b (monmap changed)... 2024-06-01T02:06:09.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:08 smithi116 ceph-mon[100606]: Reconfiguring daemon mon.b on smithi161 2024-06-01T02:06:09.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:09.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:09.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:09.913 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:09 smithi161 ceph-mon[84406]: pgmap v52: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:06:09.913 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:09.914 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:09.914 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:06:09.914 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-01T02:06:09.914 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:10.026 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:09 smithi116 ceph-mon[89502]: pgmap v52: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:06:10.026 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:10.026 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:10.027 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:06:10.027 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-01T02:06:10.027 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:10.027 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:09 smithi116 ceph-mon[100606]: pgmap v52: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:06:10.027 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:10.027 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:10.027 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-01T02:06:10.027 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-01T02:06:10.027 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:11.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:10 smithi116 ceph-mon[89502]: Reconfiguring mgr.x (monmap changed)... 2024-06-01T02:06:11.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:10 smithi116 ceph-mon[89502]: Reconfiguring daemon mgr.x on smithi161 2024-06-01T02:06:11.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:10 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:11.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:10 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:11.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:10 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-01T02:06:11.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:10 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:11.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:10 smithi116 ceph-mon[100606]: Reconfiguring mgr.x (monmap changed)... 2024-06-01T02:06:11.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:10 smithi116 ceph-mon[100606]: Reconfiguring daemon mgr.x on smithi161 2024-06-01T02:06:11.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:10 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:11.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:10 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:11.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:10 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-01T02:06:11.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:10 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:11.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:10 smithi161 ceph-mon[84406]: Reconfiguring mgr.x (monmap changed)... 2024-06-01T02:06:11.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:10 smithi161 ceph-mon[84406]: Reconfiguring daemon mgr.x on smithi161 2024-06-01T02:06:11.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:10 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:11.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:10 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:11.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:10 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-01T02:06:11.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:10 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:12.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:11 smithi161 ceph-mon[84406]: Reconfiguring osd.4 (monmap changed)... 2024-06-01T02:06:12.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:11 smithi161 ceph-mon[84406]: Reconfiguring daemon osd.4 on smithi161 2024-06-01T02:06:12.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:11 smithi161 ceph-mon[84406]: pgmap v53: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:06:12.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:11 smithi116 ceph-mon[89502]: Reconfiguring osd.4 (monmap changed)... 2024-06-01T02:06:12.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:11 smithi116 ceph-mon[89502]: Reconfiguring daemon osd.4 on smithi161 2024-06-01T02:06:12.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:11 smithi116 ceph-mon[89502]: pgmap v53: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:06:12.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:11 smithi116 ceph-mon[100606]: Reconfiguring osd.4 (monmap changed)... 2024-06-01T02:06:12.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:11 smithi116 ceph-mon[100606]: Reconfiguring daemon osd.4 on smithi161 2024-06-01T02:06:12.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:11 smithi116 ceph-mon[100606]: pgmap v53: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:06:13.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:12 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:06:13.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:12 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:13.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:12 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:13.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:12 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-01T02:06:13.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:12 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:13.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:12 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:06:13.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:12 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:13.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:12 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:13.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:12 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-01T02:06:13.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:12 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:13.178 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:12 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:06:13.178 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:12 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:13.178 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:12 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:13.178 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:12 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-01T02:06:13.178 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:12 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:13.889 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:06:13 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:06:13.505Z caller=manager.go:609 level=warn component="rule manager" group=nodes msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on(instance) group_left(nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi116\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:06:14.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:13 smithi161 ceph-mon[84406]: Reconfiguring osd.5 (monmap changed)... 2024-06-01T02:06:14.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:13 smithi161 ceph-mon[84406]: Reconfiguring daemon osd.5 on smithi161 2024-06-01T02:06:14.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:13 smithi161 ceph-mon[84406]: pgmap v54: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:06:14.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:13 smithi116 ceph-mon[89502]: Reconfiguring osd.5 (monmap changed)... 2024-06-01T02:06:14.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:13 smithi116 ceph-mon[89502]: Reconfiguring daemon osd.5 on smithi161 2024-06-01T02:06:14.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:13 smithi116 ceph-mon[89502]: pgmap v54: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:06:14.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:13 smithi116 ceph-mon[100606]: Reconfiguring osd.5 (monmap changed)... 2024-06-01T02:06:14.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:13 smithi116 ceph-mon[100606]: Reconfiguring daemon osd.5 on smithi161 2024-06-01T02:06:14.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:13 smithi116 ceph-mon[100606]: pgmap v54: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:06:15.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:15.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:15.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:14 smithi161 ceph-mon[84406]: Reconfiguring osd.6 (monmap changed)... 2024-06-01T02:06:15.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-01T02:06:15.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:15.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:14 smithi161 ceph-mon[84406]: Reconfiguring daemon osd.6 on smithi161 2024-06-01T02:06:15.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:14 smithi161 ceph-mon[84406]: pgmap v55: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:06:15.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:15.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:15.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:14 smithi116 ceph-mon[89502]: Reconfiguring osd.6 (monmap changed)... 2024-06-01T02:06:15.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-01T02:06:15.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:15.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:14 smithi116 ceph-mon[89502]: Reconfiguring daemon osd.6 on smithi161 2024-06-01T02:06:15.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:14 smithi116 ceph-mon[89502]: pgmap v55: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:06:15.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:15.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:15.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:14 smithi116 ceph-mon[100606]: Reconfiguring osd.6 (monmap changed)... 2024-06-01T02:06:15.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-01T02:06:15.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:15.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:14 smithi116 ceph-mon[100606]: Reconfiguring daemon osd.6 on smithi161 2024-06-01T02:06:15.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:14 smithi116 ceph-mon[100606]: pgmap v55: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:06:16.797 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:16.797 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:16.798 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:16 smithi161 ceph-mon[84406]: Reconfiguring osd.7 (monmap changed)... 2024-06-01T02:06:16.798 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-01T02:06:16.798 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:16.798 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:16 smithi161 ceph-mon[84406]: Reconfiguring daemon osd.7 on smithi161 2024-06-01T02:06:16.798 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:06:17.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:17.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:17.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:16 smithi116 ceph-mon[89502]: Reconfiguring osd.7 (monmap changed)... 2024-06-01T02:06:17.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-01T02:06:17.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:17.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:16 smithi116 ceph-mon[89502]: Reconfiguring daemon osd.7 on smithi161 2024-06-01T02:06:17.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:06:17.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:17.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:17.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:16 smithi116 ceph-mon[100606]: Reconfiguring osd.7 (monmap changed)... 2024-06-01T02:06:17.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-01T02:06:17.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:17.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:16 smithi116 ceph-mon[100606]: Reconfiguring daemon osd.7 on smithi161 2024-06-01T02:06:17.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:06:17.143 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:06:16 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:06:16] "GET /metrics HTTP/1.1" 200 38276 "" "Prometheus/2.33.4" 2024-06-01T02:06:17.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:17 smithi161 ceph-mon[84406]: pgmap v56: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:06:18.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:17 smithi116 ceph-mon[89502]: pgmap v56: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:06:18.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:17 smithi116 ceph-mon[100606]: pgmap v56: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:06:18.779 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:06:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:06:18.660Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"172.21.15.161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"smithi161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:06:19.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:19.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:19.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:19 smithi116 ceph-mon[89502]: Reconfiguring rgw.foo.smithi161.ashqxz (monmap changed)... 2024-06-01T02:06:19.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi161.ashqxz", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T02:06:19.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:19.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:19 smithi116 ceph-mon[89502]: Reconfiguring daemon rgw.foo.smithi161.ashqxz on smithi161 2024-06-01T02:06:19.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:19 smithi116 ceph-mon[89502]: pgmap v57: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:06:19.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:19.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:19.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:19 smithi116 ceph-mon[100606]: Reconfiguring rgw.foo.smithi161.ashqxz (monmap changed)... 2024-06-01T02:06:19.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi161.ashqxz", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T02:06:19.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:19.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:19 smithi116 ceph-mon[100606]: Reconfiguring daemon rgw.foo.smithi161.ashqxz on smithi161 2024-06-01T02:06:19.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:19 smithi116 ceph-mon[100606]: pgmap v57: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:06:19.583 INFO:teuthology.orchestra.run.smithi116.stdout:true 2024-06-01T02:06:19.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:19.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:19.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:19 smithi161 ceph-mon[84406]: Reconfiguring rgw.foo.smithi161.ashqxz (monmap changed)... 2024-06-01T02:06:19.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi161.ashqxz", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T02:06:19.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:19.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:19 smithi161 ceph-mon[84406]: Reconfiguring daemon rgw.foo.smithi161.ashqxz on smithi161 2024-06-01T02:06:19.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:19 smithi161 ceph-mon[84406]: pgmap v57: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:06:20.240 INFO:teuthology.orchestra.run.smithi116.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-01T02:06:20.241 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi116 smithi116 running 12s ago 14m - - 2024-06-01T02:06:20.241 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi161 smithi161 running 0s ago 14m - - 2024-06-01T02:06:20.241 INFO:teuthology.orchestra.run.smithi116.stdout:alertmanager.a smithi116 *:9093,9094 running (4m) 12s ago 9m 21.2M - 0.23.0 ba2b418f427c e0c297162ded 2024-06-01T02:06:20.241 INFO:teuthology.orchestra.run.smithi116.stdout:grafana.a smithi161 *:3000 running (3m) 0s ago 9m 52.3M - 8.3.5 dad864ee21e9 00b0a78ee88e 2024-06-01T02:06:20.241 INFO:teuthology.orchestra.run.smithi116.stdout:iscsi.foo.smithi116.gjuxzb smithi116 running (3m) 12s ago 7m 50.8M - 3.5 e1d6a67b021e 0e191fe3e3bb 2024-06-01T02:06:20.241 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.x smithi161 *:8443,9283,8765 running (3m) 0s ago 14m 513M - 19.0.0-2946-g3d89accf b045ac545f82 2d7fbb85e5f7 2024-06-01T02:06:20.241 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.y smithi116 *:8443,9283,8765 running (2m) 12s ago 15m 452M - 19.0.0-2946-g3d89accf b045ac545f82 bc9627c27a57 2024-06-01T02:06:20.241 INFO:teuthology.orchestra.run.smithi116.stdout:mon.a smithi116 running (94s) 12s ago 15m 50.1M 2048M 19.0.0-2946-g3d89accf b045ac545f82 b5e9b6f873ff 2024-06-01T02:06:20.241 INFO:teuthology.orchestra.run.smithi116.stdout:mon.b smithi161 running (29s) 0s ago 14m 30.0M 2048M 19.0.0-2946-g3d89accf b045ac545f82 aaa08c03cfc8 2024-06-01T02:06:20.241 INFO:teuthology.orchestra.run.smithi116.stdout:mon.c smithi116 running (45s) 12s ago 14m 37.9M 2048M 19.0.0-2946-g3d89accf b045ac545f82 bd0fb0e603fa 2024-06-01T02:06:20.241 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.a smithi116 *:9100 running (4m) 12s ago 10m 17.7M - 1.3.1 1dbe0e931976 e347748d17bb 2024-06-01T02:06:20.241 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.b smithi161 *:9100 running (4m) 0s ago 10m 17.2M - 1.3.1 1dbe0e931976 6e66182f5a1a 2024-06-01T02:06:20.241 INFO:teuthology.orchestra.run.smithi116.stdout:osd.0 smithi116 running (13m) 12s ago 13m 55.3M 4096M 17.2.0 e1d6a67b021e 0cc929b0ecad 2024-06-01T02:06:20.242 INFO:teuthology.orchestra.run.smithi116.stdout:osd.1 smithi116 running (13m) 12s ago 13m 56.9M 4096M 17.2.0 e1d6a67b021e 6ff7fc5abe58 2024-06-01T02:06:20.242 INFO:teuthology.orchestra.run.smithi116.stdout:osd.2 smithi116 running (13m) 12s ago 13m 53.9M 4096M 17.2.0 e1d6a67b021e 6aecbb10256b 2024-06-01T02:06:20.242 INFO:teuthology.orchestra.run.smithi116.stdout:osd.3 smithi116 running (12m) 12s ago 12m 55.7M 4096M 17.2.0 e1d6a67b021e a802290f5290 2024-06-01T02:06:20.242 INFO:teuthology.orchestra.run.smithi116.stdout:osd.4 smithi161 running (12m) 0s ago 12m 58.0M 4096M 17.2.0 e1d6a67b021e 70b3f89b0bd9 2024-06-01T02:06:20.242 INFO:teuthology.orchestra.run.smithi116.stdout:osd.5 smithi161 running (11m) 0s ago 11m 52.2M 4096M 17.2.0 e1d6a67b021e 68358b78a25d 2024-06-01T02:06:20.242 INFO:teuthology.orchestra.run.smithi116.stdout:osd.6 smithi161 running (11m) 0s ago 11m 52.9M 4096M 17.2.0 e1d6a67b021e 50abeb0551e7 2024-06-01T02:06:20.242 INFO:teuthology.orchestra.run.smithi116.stdout:osd.7 smithi161 running (11m) 0s ago 11m 55.7M 4096M 17.2.0 e1d6a67b021e 6004e6983735 2024-06-01T02:06:20.242 INFO:teuthology.orchestra.run.smithi116.stdout:prometheus.a smithi161 *:9095 running (2m) 0s ago 10m 74.7M - 2.33.4 514e6a882f6e e977f6ca1d73 2024-06-01T02:06:20.242 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi116.vzjrxj smithi116 *:8000 running (8m) 12s ago 8m 76.1M - 17.2.0 e1d6a67b021e df4fe3c1dbc3 2024-06-01T02:06:20.242 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi161.ashqxz smithi161 *:8000 running (8m) 0s ago 8m 77.2M - 17.2.0 e1d6a67b021e 3eecc1f3e2d3 2024-06-01T02:06:20.242 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi116.hokbmf smithi116 *:80 running (8m) 12s ago 8m 76.6M - 17.2.0 e1d6a67b021e 10950ed75dbd 2024-06-01T02:06:20.243 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi161.axjlzw smithi161 *:80 running (8m) 0s ago 8m 75.8M - 17.2.0 e1d6a67b021e 0b5d5ce47772 2024-06-01T02:06:20.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:20.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:20.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:20.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:20 smithi116 ceph-mon[89502]: from='client.64138 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:06:20.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:20 smithi116 ceph-mon[89502]: from='client.64144 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:06:20.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:20.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:20.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:20.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:20 smithi116 ceph-mon[100606]: from='client.64138 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:06:20.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:20 smithi116 ceph-mon[100606]: from='client.64144 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:06:20.636 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:06:20.637 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": { 2024-06-01T02:06:20.637 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 3 2024-06-01T02:06:20.637 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:06:20.637 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": { 2024-06-01T02:06:20.637 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-01T02:06:20.637 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:06:20.637 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": { 2024-06-01T02:06:20.637 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2024-06-01T02:06:20.637 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:06:20.637 INFO:teuthology.orchestra.run.smithi116.stdout: "rgw": { 2024-06-01T02:06:20.637 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-06-01T02:06:20.637 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:06:20.637 INFO:teuthology.orchestra.run.smithi116.stdout: "overall": { 2024-06-01T02:06:20.638 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 12, 2024-06-01T02:06:20.638 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 5 2024-06-01T02:06:20.638 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T02:06:20.638 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:06:20.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:20.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:20.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:20.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:20 smithi161 ceph-mon[84406]: from='client.64138 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:06:20.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:20 smithi161 ceph-mon[84406]: from='client.64144 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:06:20.966 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:06:20.966 INFO:teuthology.orchestra.run.smithi116.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-01T02:06:20.966 INFO:teuthology.orchestra.run.smithi116.stdout: "in_progress": true, 2024-06-01T02:06:20.966 INFO:teuthology.orchestra.run.smithi116.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-01T02:06:20.966 INFO:teuthology.orchestra.run.smithi116.stdout: "services_complete": [ 2024-06-01T02:06:20.966 INFO:teuthology.orchestra.run.smithi116.stdout: "mon", 2024-06-01T02:06:20.966 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr" 2024-06-01T02:06:20.966 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-06-01T02:06:20.966 INFO:teuthology.orchestra.run.smithi116.stdout: "progress": "5/23 daemons upgraded", 2024-06-01T02:06:20.966 INFO:teuthology.orchestra.run.smithi116.stdout: "message": "Currently upgrading mon daemons", 2024-06-01T02:06:20.966 INFO:teuthology.orchestra.run.smithi116.stdout: "is_paused": false 2024-06-01T02:06:20.967 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:06:21.365 INFO:teuthology.orchestra.run.smithi116.stdout:HEALTH_OK 2024-06-01T02:06:21.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:21 smithi161 ceph-mon[84406]: from='client.64150 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:06:21.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:21 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/1505667173' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:21.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:21 smithi161 ceph-mon[84406]: pgmap v58: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:06:21.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:21.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:21.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi161.axjlzw", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T02:06:21.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:21.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:21 smithi116 ceph-mon[89502]: from='client.64150 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:06:21.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:21 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/1505667173' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:21.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:21 smithi116 ceph-mon[89502]: pgmap v58: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:06:21.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:21.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:21.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi161.axjlzw", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T02:06:21.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:21.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:21 smithi116 ceph-mon[100606]: from='client.64150 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:06:21.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:21 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/1505667173' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:21.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:21 smithi116 ceph-mon[100606]: pgmap v58: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:06:21.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:21.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:21.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi161.axjlzw", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T02:06:21.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:22.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:22 smithi161 ceph-mon[84406]: from='client.64162 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:06:22.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:22 smithi161 ceph-mon[84406]: Reconfiguring rgw.smpl.smithi161.axjlzw (monmap changed)... 2024-06-01T02:06:22.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:22 smithi161 ceph-mon[84406]: Reconfiguring daemon rgw.smpl.smithi161.axjlzw on smithi161 2024-06-01T02:06:22.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:22 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/4256335074' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:06:22.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:22 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:06:22.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:22 smithi116 ceph-mon[89502]: from='client.64162 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:06:22.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:22 smithi116 ceph-mon[89502]: Reconfiguring rgw.smpl.smithi161.axjlzw (monmap changed)... 2024-06-01T02:06:22.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:22 smithi116 ceph-mon[89502]: Reconfiguring daemon rgw.smpl.smithi161.axjlzw on smithi161 2024-06-01T02:06:22.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:22 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/4256335074' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:06:22.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:22 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:06:22.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:22 smithi116 ceph-mon[100606]: from='client.64162 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:06:22.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:22 smithi116 ceph-mon[100606]: Reconfiguring rgw.smpl.smithi161.axjlzw (monmap changed)... 2024-06-01T02:06:22.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:22 smithi116 ceph-mon[100606]: Reconfiguring daemon rgw.smpl.smithi161.axjlzw on smithi161 2024-06-01T02:06:22.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:22 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/4256335074' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:06:22.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:22 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:06:23.843 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:06:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:06:23.505Z caller=manager.go:609 level=warn component="rule manager" group=nodes msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on(instance) group_left(nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi116\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:06:24.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[89502]: pgmap v59: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:06:24.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:24.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:24.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:24.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:24.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:24.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[89502]: Upgrade: Setting container_image for all mon 2024-06-01T02:06:24.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:24.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2024-06-01T02:06:24.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]': finished 2024-06-01T02:06:24.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2024-06-01T02:06:24.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]': finished 2024-06-01T02:06:24.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2024-06-01T02:06:24.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]': finished 2024-06-01T02:06:24.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:24.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[89502]: Upgrade: Setting container_image for all crash 2024-06-01T02:06:24.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:24.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-06-01T02:06:24.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[89502]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-06-01T02:06:24.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[100606]: pgmap v59: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:06:24.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:24.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:24.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:24.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:24.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:24.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[100606]: Upgrade: Setting container_image for all mon 2024-06-01T02:06:24.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:24.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2024-06-01T02:06:24.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]': finished 2024-06-01T02:06:24.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2024-06-01T02:06:24.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]': finished 2024-06-01T02:06:24.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2024-06-01T02:06:24.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]': finished 2024-06-01T02:06:24.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:24.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[100606]: Upgrade: Setting container_image for all crash 2024-06-01T02:06:24.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:24.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-06-01T02:06:24.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:23 smithi116 ceph-mon[100606]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-06-01T02:06:24.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:23 smithi161 ceph-mon[84406]: pgmap v59: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:06:24.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:23 smithi161 ceph-mon[84406]: Upgrade: Setting container_image for all mon 2024-06-01T02:06:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2024-06-01T02:06:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]': finished 2024-06-01T02:06:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2024-06-01T02:06:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]': finished 2024-06-01T02:06:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2024-06-01T02:06:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]': finished 2024-06-01T02:06:24.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:23 smithi161 ceph-mon[84406]: Upgrade: Setting container_image for all crash 2024-06-01T02:06:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-06-01T02:06:24.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:23 smithi161 ceph-mon[84406]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-06-01T02:06:25.116 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:24 smithi116 ceph-mon[100606]: Upgrade: osd.0 is safe to restart 2024-06-01T02:06:25.116 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:25.117 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-01T02:06:25.117 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:25.117 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:24 smithi116 ceph-mon[89502]: Upgrade: osd.0 is safe to restart 2024-06-01T02:06:25.117 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:25.117 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-01T02:06:25.117 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:25.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:24 smithi161 ceph-mon[84406]: Upgrade: osd.0 is safe to restart 2024-06-01T02:06:25.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:25.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-01T02:06:25.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:26.042 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:25 smithi116 ceph-mon[100606]: Upgrade: Updating osd.0 2024-06-01T02:06:26.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:25 smithi116 ceph-mon[100606]: Deploying daemon osd.0 on smithi116 2024-06-01T02:06:26.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:25 smithi116 ceph-mon[100606]: pgmap v60: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:06:26.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:25 smithi116 ceph-mon[89502]: Upgrade: Updating osd.0 2024-06-01T02:06:26.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:25 smithi116 ceph-mon[89502]: Deploying daemon osd.0 on smithi116 2024-06-01T02:06:26.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:25 smithi116 ceph-mon[89502]: pgmap v60: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:06:26.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:25 smithi161 ceph-mon[84406]: Upgrade: Updating osd.0 2024-06-01T02:06:26.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:25 smithi161 ceph-mon[84406]: Deploying daemon osd.0 on smithi116 2024-06-01T02:06:26.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:25 smithi161 ceph-mon[84406]: pgmap v60: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:06:27.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:06:26 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:06:26] "GET /metrics HTTP/1.1" 200 38276 "" "Prometheus/2.33.4" 2024-06-01T02:06:27.333 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:27 smithi116 ceph-mon[100606]: pgmap v61: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:06:27.334 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:27 smithi116 ceph-mon[89502]: pgmap v61: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:06:27.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:27 smithi161 ceph-mon[84406]: pgmap v61: 161 pgs: 161 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:06:27.930 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:27 smithi116 systemd[1]: Stopping Ceph osd.0 for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:06:28.293 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:27 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0[33260]: 2024-06-01T02:06:27.933+0000 7fab790bc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-01T02:06:28.293 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:27 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0[33260]: 2024-06-01T02:06:27.933+0000 7fab790bc700 -1 osd.0 114 *** Got signal Terminated *** 2024-06-01T02:06:28.293 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:27 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0[33260]: 2024-06-01T02:06:27.933+0000 7fab790bc700 -1 osd.0 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-01T02:06:28.622 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:28 smithi116 ceph-mon[100606]: osd.0 marked itself down and dead 2024-06-01T02:06:28.623 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:28.623 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:28.623 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:28.623 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:28 smithi116 ceph-mon[89502]: osd.0 marked itself down and dead 2024-06-01T02:06:28.623 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:28.623 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:28.623 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:28.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:28 smithi161 ceph-mon[84406]: osd.0 marked itself down and dead 2024-06-01T02:06:28.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:28.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:28.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:28.897 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:28 smithi116 podman[106785]: 2024-06-01 02:06:28.627598171 +0000 UTC m=+0.793069880 container died 0cc929b0ecadfe46f295580412ac2097efeaee8bcdd17a39957e6d06e5f48988 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0, ceph=True, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, GIT_CLEAN=True, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, version=8, release=754, name=centos-stream, vendor=Red Hat, Inc., io.k8s.display-name=CentOS Stream 8, architecture=x86_64, maintainer=Guillaume Abrioux , io.buildah.version=1.19.8, vcs-type=git, GIT_BRANCH=HEAD, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, distribution-scope=public, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, RELEASE=HEAD, CEPH_POINT_RELEASE=-17.2.0, GIT_REPO=https://github.com/ceph/ceph-container.git, com.redhat.component=centos-stream-container, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, build-date=2022-05-03T08:36:31.336870, io.openshift.expose-services=, io.openshift.tags=base centos centos-stream, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly.) 2024-06-01T02:06:28.944 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:06:28 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:06:28.660Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"172.21.15.161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"smithi161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:06:29.151 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:29 smithi116 podman[106785]: 2024-06-01 02:06:29.134419622 +0000 UTC m=+1.299891344 container cleanup 0cc929b0ecadfe46f295580412ac2097efeaee8bcdd17a39957e6d06e5f48988 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0, version=8, ceph=True, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, GIT_CLEAN=True, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, architecture=x86_64, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, CEPH_POINT_RELEASE=-17.2.0, vendor=Red Hat, Inc., distribution-scope=public, com.redhat.component=centos-stream-container, io.buildah.version=1.19.8, vcs-type=git, release=754, GIT_BRANCH=HEAD, RELEASE=HEAD, io.k8s.display-name=CentOS Stream 8, name=centos-stream, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., maintainer=Guillaume Abrioux , io.openshift.expose-services=, GIT_REPO=https://github.com/ceph/ceph-container.git, build-date=2022-05-03T08:36:31.336870, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, io.openshift.tags=base centos centos-stream) 2024-06-01T02:06:29.151 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:29 smithi116 bash[106785]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0 2024-06-01T02:06:29.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:29 smithi161 ceph-mon[84406]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:06:29.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:29 smithi161 ceph-mon[84406]: osdmap e115: 8 total, 7 up, 8 in 2024-06-01T02:06:29.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:29 smithi161 ceph-mon[84406]: pgmap v63: 161 pgs: 21 stale+active+clean, 140 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:06:29.722 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:29 smithi116 ceph-mon[89502]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:06:29.722 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:29 smithi116 ceph-mon[89502]: osdmap e115: 8 total, 7 up, 8 in 2024-06-01T02:06:29.722 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:29 smithi116 ceph-mon[89502]: pgmap v63: 161 pgs: 21 stale+active+clean, 140 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:06:29.722 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:29 smithi116 ceph-mon[100606]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:06:29.722 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:29 smithi116 ceph-mon[100606]: osdmap e115: 8 total, 7 up, 8 in 2024-06-01T02:06:29.722 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:29 smithi116 ceph-mon[100606]: pgmap v63: 161 pgs: 21 stale+active+clean, 140 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:06:29.723 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:29 smithi116 podman[106825]: 2024-06-01 02:06:29.411542603 +0000 UTC m=+0.778233189 container remove 0cc929b0ecadfe46f295580412ac2097efeaee8bcdd17a39957e6d06e5f48988 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0, io.buildah.version=1.19.8, io.k8s.display-name=CentOS Stream 8, RELEASE=HEAD, vcs-type=git, com.redhat.component=centos-stream-container, io.openshift.expose-services=, CEPH_POINT_RELEASE=-17.2.0, build-date=2022-05-03T08:36:31.336870, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, ceph=True, GIT_BRANCH=HEAD, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, architecture=x86_64, version=8, vendor=Red Hat, Inc., GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , distribution-scope=public, io.openshift.tags=base centos centos-stream, name=centos-stream, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, release=754, GIT_CLEAN=True) 2024-06-01T02:06:30.043 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:29 smithi116 podman[106914]: 2024-06-01 02:06:29.62941064 +0000 UTC m=+0.017129835 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:06:30.043 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:29 smithi116 podman[106914]: 2024-06-01 02:06:29.728894737 +0000 UTC m=+0.116613928 container create 6183adf47d312edb504e1582c0f0818a05b4a816c9dd7e8f08657a8326e303f0 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-06-01T02:06:30.354 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:30 smithi116 ceph-mon[89502]: osdmap e116: 8 total, 7 up, 8 in 2024-06-01T02:06:30.355 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:30 smithi116 ceph-mon[100606]: osdmap e116: 8 total, 7 up, 8 in 2024-06-01T02:06:30.355 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:30 smithi116 podman[106914]: 2024-06-01 02:06:30.15027365 +0000 UTC m=+0.537992857 container init 6183adf47d312edb504e1582c0f0818a05b4a816c9dd7e8f08657a8326e303f0 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc) 2024-06-01T02:06:30.355 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:30 smithi116 podman[106914]: 2024-06-01 02:06:30.153819274 +0000 UTC m=+0.541538472 container start 6183adf47d312edb504e1582c0f0818a05b4a816c9dd7e8f08657a8326e303f0 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-01T02:06:30.356 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:30 smithi116 podman[106914]: 2024-06-01 02:06:30.233147295 +0000 UTC m=+0.620866504 container attach 6183adf47d312edb504e1582c0f0818a05b4a816c9dd7e8f08657a8326e303f0 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, ceph=True, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-01T02:06:30.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:30 smithi161 ceph-mon[84406]: osdmap e116: 8 total, 7 up, 8 in 2024-06-01T02:06:31.018 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:30 smithi116 podman[106914]: 2024-06-01 02:06:30.681038364 +0000 UTC m=+1.068757560 container died 6183adf47d312edb504e1582c0f0818a05b4a816c9dd7e8f08657a8326e303f0 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-06-01T02:06:31.572 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:31 smithi116 ceph-mon[89502]: pgmap v65: 161 pgs: 21 stale+active+clean, 140 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-01T02:06:31.573 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1c", "id": [4, 0]}]: dispatch 2024-06-01T02:06:31.573 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.0", "id": [4, 6]}]: dispatch 2024-06-01T02:06:31.573 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.4", "id": [5, 3]}]: dispatch 2024-06-01T02:06:31.573 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.5", "id": [1, 2]}]: dispatch 2024-06-01T02:06:31.573 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.6", "id": [5, 0]}]: dispatch 2024-06-01T02:06:31.573 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.f", "id": [4, 2]}]: dispatch 2024-06-01T02:06:31.573 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.10", "id": [7, 0]}]: dispatch 2024-06-01T02:06:31.574 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.14", "id": [7, 0]}]: dispatch 2024-06-01T02:06:31.574 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1d", "id": [1, 6]}]: dispatch 2024-06-01T02:06:31.574 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1f", "id": [4, 0]}]: dispatch 2024-06-01T02:06:31.574 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:31.574 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:06:31.574 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:31 smithi116 ceph-mon[100606]: pgmap v65: 161 pgs: 21 stale+active+clean, 140 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-01T02:06:31.575 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1c", "id": [4, 0]}]: dispatch 2024-06-01T02:06:31.575 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.0", "id": [4, 6]}]: dispatch 2024-06-01T02:06:31.575 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.4", "id": [5, 3]}]: dispatch 2024-06-01T02:06:31.575 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.5", "id": [1, 2]}]: dispatch 2024-06-01T02:06:31.575 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.6", "id": [5, 0]}]: dispatch 2024-06-01T02:06:31.575 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.f", "id": [4, 2]}]: dispatch 2024-06-01T02:06:31.575 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.10", "id": [7, 0]}]: dispatch 2024-06-01T02:06:31.575 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.14", "id": [7, 0]}]: dispatch 2024-06-01T02:06:31.576 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1d", "id": [1, 6]}]: dispatch 2024-06-01T02:06:31.576 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1f", "id": [4, 0]}]: dispatch 2024-06-01T02:06:31.576 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:31.576 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:06:31.576 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:31 smithi116 podman[106972]: 2024-06-01 02:06:31.347982835 +0000 UTC m=+0.888507800 container remove 6183adf47d312edb504e1582c0f0818a05b4a816c9dd7e8f08657a8326e303f0 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-01T02:06:31.576 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:31 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.0.service: Deactivated successfully. 2024-06-01T02:06:31.576 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:31 smithi116 systemd[1]: Stopped Ceph osd.0 for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:06:31.577 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:31 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.0.service: Consumed 7.929s CPU time. 2024-06-01T02:06:31.577 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:31 smithi116 systemd[1]: Starting Ceph osd.0 for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:06:31.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:31 smithi161 ceph-mon[84406]: pgmap v65: 161 pgs: 21 stale+active+clean, 140 active+clean; 585 KiB data, 100 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-01T02:06:31.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1c", "id": [4, 0]}]: dispatch 2024-06-01T02:06:31.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.0", "id": [4, 6]}]: dispatch 2024-06-01T02:06:31.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.4", "id": [5, 3]}]: dispatch 2024-06-01T02:06:31.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.5", "id": [1, 2]}]: dispatch 2024-06-01T02:06:31.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.6", "id": [5, 0]}]: dispatch 2024-06-01T02:06:31.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.f", "id": [4, 2]}]: dispatch 2024-06-01T02:06:31.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.10", "id": [7, 0]}]: dispatch 2024-06-01T02:06:31.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.14", "id": [7, 0]}]: dispatch 2024-06-01T02:06:31.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1d", "id": [1, 6]}]: dispatch 2024-06-01T02:06:31.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1f", "id": [4, 0]}]: dispatch 2024-06-01T02:06:31.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:31.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:06:32.533 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1c", "id": [4, 0]}]': finished 2024-06-01T02:06:32.533 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.0", "id": [4, 6]}]': finished 2024-06-01T02:06:32.533 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.4", "id": [5, 3]}]': finished 2024-06-01T02:06:32.533 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.5", "id": [1, 2]}]': finished 2024-06-01T02:06:32.534 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.6", "id": [5, 0]}]': finished 2024-06-01T02:06:32.534 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.f", "id": [4, 2]}]': finished 2024-06-01T02:06:32.534 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.10", "id": [7, 0]}]': finished 2024-06-01T02:06:32.534 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.14", "id": [7, 0]}]': finished 2024-06-01T02:06:32.534 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1d", "id": [1, 6]}]': finished 2024-06-01T02:06:32.534 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1f", "id": [4, 0]}]': finished 2024-06-01T02:06:32.534 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:32 smithi116 ceph-mon[89502]: osdmap e117: 8 total, 7 up, 8 in 2024-06-01T02:06:32.534 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:32 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:06:32.534 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1c", "id": [4, 0]}]': finished 2024-06-01T02:06:32.535 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.0", "id": [4, 6]}]': finished 2024-06-01T02:06:32.535 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.4", "id": [5, 3]}]': finished 2024-06-01T02:06:32.535 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.5", "id": [1, 2]}]': finished 2024-06-01T02:06:32.535 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.6", "id": [5, 0]}]': finished 2024-06-01T02:06:32.535 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.f", "id": [4, 2]}]': finished 2024-06-01T02:06:32.535 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.10", "id": [7, 0]}]': finished 2024-06-01T02:06:32.535 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.14", "id": [7, 0]}]': finished 2024-06-01T02:06:32.535 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1d", "id": [1, 6]}]': finished 2024-06-01T02:06:32.535 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1f", "id": [4, 0]}]': finished 2024-06-01T02:06:32.535 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:32 smithi116 ceph-mon[100606]: osdmap e117: 8 total, 7 up, 8 in 2024-06-01T02:06:32.535 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:32 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:06:32.536 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:32 smithi116 podman[107071]: 2024-06-01 02:06:32.124222909 +0000 UTC m=+0.017209884 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:06:32.536 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:32 smithi116 podman[107071]: 2024-06-01 02:06:32.226828116 +0000 UTC m=+0.119815069 container create 061151e1ee1f0207e6ac4d43925579a4640b743acdae4bbeec9884f0fe41e25c (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-activate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-06-01T02:06:32.536 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:32 smithi116 podman[107071]: 2024-06-01 02:06:32.446200831 +0000 UTC m=+0.339187800 container init 061151e1ee1f0207e6ac4d43925579a4640b743acdae4bbeec9884f0fe41e25c (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-activate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-06-01T02:06:32.536 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:32 smithi116 podman[107071]: 2024-06-01 02:06:32.450400866 +0000 UTC m=+0.343387838 container start 061151e1ee1f0207e6ac4d43925579a4640b743acdae4bbeec9884f0fe41e25c (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-activate, ceph=True, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-06-01T02:06:32.536 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:32 smithi116 podman[107071]: 2024-06-01 02:06:32.537788548 +0000 UTC m=+0.430775529 container attach 061151e1ee1f0207e6ac4d43925579a4640b743acdae4bbeec9884f0fe41e25c (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-activate, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-01T02:06:32.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1c", "id": [4, 0]}]': finished 2024-06-01T02:06:32.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.0", "id": [4, 6]}]': finished 2024-06-01T02:06:32.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.4", "id": [5, 3]}]': finished 2024-06-01T02:06:32.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.5", "id": [1, 2]}]': finished 2024-06-01T02:06:32.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.6", "id": [5, 0]}]': finished 2024-06-01T02:06:32.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.f", "id": [4, 2]}]': finished 2024-06-01T02:06:32.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.10", "id": [7, 0]}]': finished 2024-06-01T02:06:32.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.14", "id": [7, 0]}]': finished 2024-06-01T02:06:32.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1d", "id": [1, 6]}]': finished 2024-06-01T02:06:32.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1f", "id": [4, 0]}]': finished 2024-06-01T02:06:32.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:32 smithi161 ceph-mon[84406]: osdmap e117: 8 total, 7 up, 8 in 2024-06-01T02:06:32.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:32 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:06:33.225 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:32 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-activate[107086]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-06-01T02:06:33.225 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:32 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-activate[107086]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-06-01T02:06:33.225 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:32 smithi116 bash[107071]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-06-01T02:06:33.225 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:32 smithi116 bash[107071]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-06-01T02:06:33.225 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:32 smithi116 bash[107071]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-06-01T02:06:33.225 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:32 smithi116 bash[107071]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-06-01T02:06:33.225 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:32 smithi116 bash[107071]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-06-01T02:06:33.225 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:32 smithi116 bash[107071]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-06-01T02:06:33.225 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:32 smithi116 bash[107071]: --> ceph-volume raw activate successful for osd ID: 0 2024-06-01T02:06:33.226 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:32 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-activate[107086]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-06-01T02:06:33.226 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:32 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-activate[107086]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-06-01T02:06:33.226 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:32 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-activate[107086]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-06-01T02:06:33.226 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:32 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-activate[107086]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-06-01T02:06:33.226 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:32 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-activate[107086]: --> ceph-volume raw activate successful for osd ID: 0 2024-06-01T02:06:33.226 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:32 smithi116 podman[107071]: 2024-06-01 02:06:32.927388891 +0000 UTC m=+0.820375854 container died 061151e1ee1f0207e6ac4d43925579a4640b743acdae4bbeec9884f0fe41e25c (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-activate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-06-01T02:06:33.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:33 smithi116 ceph-mon[89502]: osdmap e118: 8 total, 7 up, 8 in 2024-06-01T02:06:33.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:33 smithi116 ceph-mon[89502]: pgmap v68: 161 pgs: 8 peering, 21 active+undersized+degraded, 35 active+undersized, 96 active+clean, 1 stale+active+clean; 585 KiB data, 101 MiB used, 715 GiB / 715 GiB avail; 76/726 objects degraded (10.468%) 2024-06-01T02:06:33.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:33 smithi116 ceph-mon[89502]: osdmap e119: 8 total, 7 up, 8 in 2024-06-01T02:06:33.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:33 smithi116 ceph-mon[100606]: osdmap e118: 8 total, 7 up, 8 in 2024-06-01T02:06:33.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:33 smithi116 ceph-mon[100606]: pgmap v68: 161 pgs: 8 peering, 21 active+undersized+degraded, 35 active+undersized, 96 active+clean, 1 stale+active+clean; 585 KiB data, 101 MiB used, 715 GiB / 715 GiB avail; 76/726 objects degraded (10.468%) 2024-06-01T02:06:33.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:33 smithi116 ceph-mon[100606]: osdmap e119: 8 total, 7 up, 8 in 2024-06-01T02:06:33.694 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:06:33 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:06:33.505Z caller=manager.go:609 level=warn component="rule manager" group=nodes msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on(instance) group_left(nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi116\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:06:33.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:33 smithi161 ceph-mon[84406]: osdmap e118: 8 total, 7 up, 8 in 2024-06-01T02:06:33.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:33 smithi161 ceph-mon[84406]: pgmap v68: 161 pgs: 8 peering, 21 active+undersized+degraded, 35 active+undersized, 96 active+clean, 1 stale+active+clean; 585 KiB data, 101 MiB used, 715 GiB / 715 GiB avail; 76/726 objects degraded (10.468%) 2024-06-01T02:06:33.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:33 smithi161 ceph-mon[84406]: osdmap e119: 8 total, 7 up, 8 in 2024-06-01T02:06:33.928 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:33 smithi116 podman[107217]: 2024-06-01 02:06:33.575702254 +0000 UTC m=+0.642288778 container remove 061151e1ee1f0207e6ac4d43925579a4640b743acdae4bbeec9884f0fe41e25c (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-06-01T02:06:33.928 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:33 smithi116 podman[107278]: 2024-06-01 02:06:33.686856892 +0000 UTC m=+0.016994154 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:06:33.928 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:33 smithi116 podman[107278]: 2024-06-01 02:06:33.809357349 +0000 UTC m=+0.139494641 container create fd69ed064e41e01ff2870debc06157ed3bd26b9c1a067eebf838e446a24f44c1 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-06-01T02:06:34.331 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:34 smithi116 podman[107278]: 2024-06-01 02:06:34.062331051 +0000 UTC m=+0.392468299 container init fd69ed064e41e01ff2870debc06157ed3bd26b9c1a067eebf838e446a24f44c1 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-06-01T02:06:34.331 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:34 smithi116 podman[107278]: 2024-06-01 02:06:34.065534192 +0000 UTC m=+0.395671452 container start fd69ed064e41e01ff2870debc06157ed3bd26b9c1a067eebf838e446a24f44c1 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=) 2024-06-01T02:06:34.331 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:34 smithi116 ceph-osd[107298]: -- 172.21.15.116:0/2977951857 <== mon.0 v2:172.21.15.116:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e9bc283680 con 0x55e9bc24cc00 2024-06-01T02:06:34.331 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:34 smithi116 bash[107278]: fd69ed064e41e01ff2870debc06157ed3bd26b9c1a067eebf838e446a24f44c1 2024-06-01T02:06:34.331 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:34 smithi116 systemd[1]: Started Ceph osd.0 for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:06:34.603 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:34 smithi116 ceph-mon[89502]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-06-01T02:06:34.603 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:34 smithi116 ceph-mon[89502]: Health check failed: Degraded data redundancy: 76/726 objects degraded (10.468%), 21 pgs degraded (PG_DEGRADED) 2024-06-01T02:06:34.603 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:34 smithi116 ceph-mon[100606]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-06-01T02:06:34.604 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:34 smithi116 ceph-mon[100606]: Health check failed: Degraded data redundancy: 76/726 objects degraded (10.468%), 21 pgs degraded (PG_DEGRADED) 2024-06-01T02:06:34.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:34 smithi161 ceph-mon[84406]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-06-01T02:06:34.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:34 smithi161 ceph-mon[84406]: Health check failed: Degraded data redundancy: 76/726 objects degraded (10.468%), 21 pgs degraded (PG_DEGRADED) 2024-06-01T02:06:34.920 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:34 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0[107294]: 2024-06-01T02:06:34.607+0000 7f6b3d79c7c0 -1 Falling back to public interface 2024-06-01T02:06:35.481 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:35.481 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:35.481 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:35.481 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:35.481 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:35.481 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:35.481 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:35.481 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:35.481 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:35.482 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:35.482 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:35.482 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:35.482 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:35.482 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:35.482 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:35.482 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:35.482 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:35.482 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:35.482 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:35.482 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:35.483 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:35.483 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:35.483 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:35.483 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:35.483 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: pgmap v70: 161 pgs: 8 peering, 21 active+undersized+degraded, 35 active+undersized, 96 active+clean, 1 stale+active+clean; 585 KiB data, 101 MiB used, 715 GiB / 715 GiB avail; 76/726 objects degraded (10.468%) 2024-06-01T02:06:35.483 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:35.483 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:35.483 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:35.483 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:35.483 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:35.484 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:35.484 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:35.484 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:35.484 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:35.484 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:35.484 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:35.484 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:35.484 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:35.484 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:35.484 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:35.484 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:35.484 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:35.484 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:35.485 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:35.485 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:35.485 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:35.485 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:35.485 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:35.485 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:35.485 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:35.485 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:35.485 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:35.485 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: pgmap v70: 161 pgs: 8 peering, 21 active+undersized+degraded, 35 active+undersized, 96 active+clean, 1 stale+active+clean; 585 KiB data, 101 MiB used, 715 GiB / 715 GiB avail; 76/726 objects degraded (10.468%) 2024-06-01T02:06:35.485 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:35.485 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:35.485 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:35.486 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:35.486 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:35.486 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:35.486 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:35.486 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:35.486 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:35.486 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:35.487 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:35.487 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:35.487 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:35.487 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:35.487 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:35.487 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:35.487 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:35.487 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:35.487 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:35.487 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:35.487 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:35.487 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:35.488 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:35.488 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:35.488 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:35.488 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:35.488 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:35.488 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:35.488 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:35.488 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:35.488 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:35.488 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:35.489 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:35.489 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:35.489 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:35.489 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:35.489 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:35.489 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:35.489 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:35.489 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:35.490 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:35.490 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:35.490 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:35.490 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:35.490 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:35.490 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:35.491 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:35.491 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:35.491 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:35.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:35.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:35.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:35.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:35.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:35.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:35.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:35.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:35.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:35.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:35.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:35.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:35.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:35.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:35.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:35.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:35.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:35.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:35.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:35.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:35.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:35.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:35.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:35.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:35.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: pgmap v70: 161 pgs: 8 peering, 21 active+undersized+degraded, 35 active+undersized, 96 active+clean, 1 stale+active+clean; 585 KiB data, 101 MiB used, 715 GiB / 715 GiB avail; 76/726 objects degraded (10.468%) 2024-06-01T02:06:35.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:35.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:35.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:35.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:35.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:35.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:35.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:35.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:35.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:35.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:35.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:35.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:35.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:35.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:35.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:35.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:35.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:35.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:35.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:35.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:35.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:35.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:35.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:35.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:35.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:35.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:36.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:36.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:36.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:36.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:36.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:36.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:36.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:36.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:36.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:36.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:36.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:36.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:36.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:36.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:36.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:36.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:36.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:36.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:36.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:36.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:36.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:36.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:36.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:36.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:36.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:36.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:36.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:36.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:36.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:36.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:36.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:36.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:36.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:36.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:36.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:36.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:36.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:36.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:36.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:36.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:36.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:36.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:36.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:36.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:36.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:36.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:36.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:36.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:36.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:36.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:36.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:36.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:36.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:36.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:36.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:36.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:36.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:36.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:36.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:36.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:36.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:36.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:36.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:36.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:36.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:36.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:36.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:36.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:36.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:36.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:36.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:36.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:36.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:36.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:36.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:36.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:36.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:36.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:36.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:36.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:36.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:36.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:36.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:36.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:36.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:36.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:36.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:36.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:36.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:36.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:36.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:36.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:36.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:36.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:36.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:36.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:36.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:36.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:36.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:36.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:36.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:36.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:36.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:36.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:36.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:36.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:36.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:36.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:36.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:36.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:36.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:36.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:36.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:36.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:36.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:36.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:36.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:36.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:36.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:36.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:36.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:36.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:36.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:36.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:36.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:36.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:36.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:36.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:36.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:36.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:36.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:36.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:36.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:36.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:36.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:36.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:36.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:36.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:36.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:36.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:36.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:36.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:36.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:36.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:36.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:36.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:36.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:36.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:36.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:36.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:36.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:36.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:36.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:36.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:36.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:36.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:36.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:36.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:36.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:36.804 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:36.804 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:36.804 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:36.804 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:36.804 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:36.804 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:37.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:06:36 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:06:36] "GET /metrics HTTP/1.1" 200 38328 "" "Prometheus/2.33.4" 2024-06-01T02:06:37.503 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:37.504 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:37.504 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:37.504 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:37.504 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:37.504 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:37.504 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:37.504 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:37.504 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:37.504 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:37.504 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:37.504 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:37.504 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:37.504 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:37.504 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:37.505 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:37.505 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:37.505 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:37.505 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:37.505 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:37.505 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:37.505 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:37.505 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:37.505 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:37.505 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:37.505 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:37.505 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:37.505 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: pgmap v71: 161 pgs: 7 active+clean+remapped, 21 active+undersized+degraded, 35 active+undersized, 98 active+clean; 585 KiB data, 101 MiB used, 715 GiB / 715 GiB avail; 170 B/s rd, 0 op/s; 76/726 objects degraded (10.468%); 9/726 objects misplaced (1.240%); 0 B/s, 0 objects/s recovering 2024-06-01T02:06:37.506 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:37.506 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:37.506 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:37.506 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:37.506 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:37.506 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:37.506 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:37.506 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:37.507 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:37.507 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:37.507 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:37.507 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:37.507 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:37.507 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:37.507 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:37.507 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:37.507 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:37.507 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:37.507 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:37.507 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:37.507 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:37.508 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:37.508 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:37.508 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:37.508 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:37.508 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:37.508 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:37.508 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:37.543 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0[107294]: 2024-06-01T02:06:37.356+0000 7f6b3d79c7c0 -1 osd.0 0 read_superblock omap replica is missing. 2024-06-01T02:06:37.543 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0[107294]: 2024-06-01T02:06:37.387+0000 7f6b3d79c7c0 -1 osd.0 114 log_to_monitors true 2024-06-01T02:06:37.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:37.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:37.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:37.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:37.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:37.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:37.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:37.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:37.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:37.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:37.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:37.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:37.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:37.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:37.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:37.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:37.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:37.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:37.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:37.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:37.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:37.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:37.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:37.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:37.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:37.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:37.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:37.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: pgmap v71: 161 pgs: 7 active+clean+remapped, 21 active+undersized+degraded, 35 active+undersized, 98 active+clean; 585 KiB data, 101 MiB used, 715 GiB / 715 GiB avail; 170 B/s rd, 0 op/s; 76/726 objects degraded (10.468%); 9/726 objects misplaced (1.240%); 0 B/s, 0 objects/s recovering 2024-06-01T02:06:37.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:37.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:37.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:37.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:37.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:37.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:37.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:37.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:37.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:37.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:37.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:37.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:37.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:37.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:37.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:37.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:37.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:37.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:37.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:37.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:37.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:37.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:37.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:37.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:37.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:37.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:37.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:37.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: pgmap v71: 161 pgs: 7 active+clean+remapped, 21 active+undersized+degraded, 35 active+undersized, 98 active+clean; 585 KiB data, 101 MiB used, 715 GiB / 715 GiB avail; 170 B/s rd, 0 op/s; 76/726 objects degraded (10.468%); 9/726 objects misplaced (1.240%); 0 B/s, 0 objects/s recovering 2024-06-01T02:06:37.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:37.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:37.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:37.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:37.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:37.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:37.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:37.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:37.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:37.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:37.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:37.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:37.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:37.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:37.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:37.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:37.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:37.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:37.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:37.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:37.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:37.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:37.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:37.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:37.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:37.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:37.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:37.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:37.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:37.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:37.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:37.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:37.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:37.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:37.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:37.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:37.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:37.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:37.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:37.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:37.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:37.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:37.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:37.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:37.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:37.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:37.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:37.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:37.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:37.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:37.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:37.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:37.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:37.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:37.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:37.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:38.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:38.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:38.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:38.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:38.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='osd.0 [v2:172.21.15.116:6800/3889754334,v1:172.21.15.116:6801/3889754334]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-01T02:06:38.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:38.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:38.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:38.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:38.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:38.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:38.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:38.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:38.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:38.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:38.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:38.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:38.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:38.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:38.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:38.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:38.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:38.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:38.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:38.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:38.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:38.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:38.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:38.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:38.588 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-06-01T02:06:38.588 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:38.588 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:38.588 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:38.588 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:38.588 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:38.588 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:38.588 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:38.589 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:38.589 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:38.589 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:38.589 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:38.589 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:38.589 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:38.589 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:38.589 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:38.589 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:38.589 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:38.589 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:38.589 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:38.589 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:38.589 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:38.590 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:38.590 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:38.590 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:38.590 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:38.590 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:38.590 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:38.590 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:38.590 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:38.590 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:38.609 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0[107294]: 2024-06-01T02:06:38.413+0000 7f6b2ebf3700 -1 osd.0 114 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-01T02:06:38.610 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:38.610 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:38.610 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:38.610 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:38.610 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='osd.0 [v2:172.21.15.116:6800/3889754334,v1:172.21.15.116:6801/3889754334]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-01T02:06:38.610 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:38.610 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:38.610 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:38.610 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:38.610 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:38.610 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:38.610 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:38.610 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:38.611 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:38.611 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:38.611 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:38.611 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:38.611 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:38.611 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:38.611 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:38.611 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:38.611 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:38.611 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:38.611 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:38.611 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:38.611 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:38.612 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:38.612 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:38.612 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:38.612 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:38.612 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:38.612 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:38.612 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:38.612 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='osd.0 [v2:172.21.15.116:6800/3889754334,v1:172.21.15.116:6801/3889754334]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-01T02:06:38.612 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:38.612 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:38.612 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:38.613 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:38.613 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:38.613 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:38.613 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:38.613 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:38.613 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:38.613 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:38.613 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:38.613 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:38.613 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:38.613 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:38.613 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:38.613 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:38.613 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:38.613 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:38.613 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:38.614 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:38.614 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:38.614 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:38.614 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:38.614 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:38.614 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-06-01T02:06:38.614 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:38.614 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:38.615 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:38.615 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:38.615 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:38.615 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:38.615 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:38.615 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:38.615 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:38.615 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:38.615 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:38.615 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:38.615 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:38.615 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:38.615 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:38.615 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:38.616 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:38.616 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:38.616 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:38.616 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:38.616 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:38.616 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:38.616 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:38.616 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:38.616 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:38.617 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:38.617 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:38.617 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:38.617 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:38.617 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:38.617 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-06-01T02:06:38.617 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:38.617 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:38.617 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:38.617 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:38.617 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:38.618 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:38.618 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:38.618 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:38.618 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:38.618 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:38.618 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:38.618 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:38.618 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:38.618 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:38.618 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:38.618 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:38.618 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:38.618 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:38.619 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:38.619 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:38.619 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:38.619 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:38.619 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:38.619 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:38.619 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:38.619 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:38.619 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:38.619 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:38.619 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:38.619 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:38.837 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:06:38.659Z caller=alerting.go:353 level=warn component="rule manager" alert="unsupported value type" msg="Expanding alert template failed" err="error executing template __alert_CephOSDDown: template: __alert_CephOSDDown:1:358: executing \"__alert_CephOSDDown\" at : error calling query: found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"172.21.15.161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"smithi161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}];many-to-many matching not allowed: matching labels must be unique on one side" data="unsupported value type" 2024-06-01T02:06:38.837 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:06:38 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:06:38.660Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"172.21.15.161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"smithi161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:06:39.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:39.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:39.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:39.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:39.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:39.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:39.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='osd.0 [v2:172.21.15.116:6800/3889754334,v1:172.21.15.116:6801/3889754334]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-01T02:06:39.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: osdmap e120: 8 total, 7 up, 8 in 2024-06-01T02:06:39.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='osd.0 [v2:172.21.15.116:6800/3889754334,v1:172.21.15.116:6801/3889754334]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T02:06:39.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:39.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:39.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:39.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:39.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:39.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:39.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:39.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:39.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:39.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:39.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:39.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:39.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:39.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:39.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:39.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:39.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:39.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:39.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: pgmap v73: 161 pgs: 7 active+clean+remapped, 21 active+undersized+degraded, 35 active+undersized, 98 active+clean; 585 KiB data, 117 MiB used, 715 GiB / 715 GiB avail; 789 B/s rd, 0 op/s; 76/726 objects degraded (10.468%); 9/726 objects misplaced (1.240%); 0 B/s, 0 objects/s recovering 2024-06-01T02:06:39.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:39.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:39.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:39.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:39.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:39.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:39.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:39.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:39.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:39.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:39.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:39.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:39.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:39.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:39.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:39.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:39.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:39.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:39.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:39.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:39.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:39.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='osd.0 [v2:172.21.15.116:6800/3889754334,v1:172.21.15.116:6801/3889754334]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-01T02:06:39.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: osdmap e120: 8 total, 7 up, 8 in 2024-06-01T02:06:39.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='osd.0 [v2:172.21.15.116:6800/3889754334,v1:172.21.15.116:6801/3889754334]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T02:06:39.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:39.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:39.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:39.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:39.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:39.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:39.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:39.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:39.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:39.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:39.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:39.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:39.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:39.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:39.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:39.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:39.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:39.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:39.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: pgmap v73: 161 pgs: 7 active+clean+remapped, 21 active+undersized+degraded, 35 active+undersized, 98 active+clean; 585 KiB data, 117 MiB used, 715 GiB / 715 GiB avail; 789 B/s rd, 0 op/s; 76/726 objects degraded (10.468%); 9/726 objects misplaced (1.240%); 0 B/s, 0 objects/s recovering 2024-06-01T02:06:39.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:39.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:39.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:39.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:39.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:39.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:39.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='osd.0 [v2:172.21.15.116:6800/3889754334,v1:172.21.15.116:6801/3889754334]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-01T02:06:39.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: osdmap e120: 8 total, 7 up, 8 in 2024-06-01T02:06:39.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='osd.0 [v2:172.21.15.116:6800/3889754334,v1:172.21.15.116:6801/3889754334]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T02:06:39.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:39.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:39.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:39.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:39.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:39.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:39.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:39.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:39.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:39.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:39.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:39.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:39.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:39.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:39.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:39.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:39.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:39.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:39.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: pgmap v73: 161 pgs: 7 active+clean+remapped, 21 active+undersized+degraded, 35 active+undersized, 98 active+clean; 585 KiB data, 117 MiB used, 715 GiB / 715 GiB avail; 789 B/s rd, 0 op/s; 76/726 objects degraded (10.468%); 9/726 objects misplaced (1.240%); 0 B/s, 0 objects/s recovering 2024-06-01T02:06:39.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:39.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:39.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:39.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:39.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:39.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:39.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:39.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:39.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:39.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:39.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:39.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:39.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:39.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:39.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:39.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:39.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:39.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:39.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:39.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:39.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:39.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:39.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:39.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:39.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:39.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:39.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:39.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:39.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:39.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.403 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:40.403 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:40.403 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:40.403 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:06:40.403 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: osd.0 [v2:172.21.15.116:6800/3889754334,v1:172.21.15.116:6801/3889754334] boot 2024-06-01T02:06:40.404 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: osdmap e121: 8 total, 8 up, 8 in 2024-06-01T02:06:40.404 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T02:06:40.404 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:40.404 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:40.404 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:40.405 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:40.405 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.405 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:40.405 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.405 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.405 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.406 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:40.406 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:40.406 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:40.406 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:40.406 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.406 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:40.406 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:40.406 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:40.407 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:40.407 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:40.407 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.407 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:40.407 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:40.408 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:40.408 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:40.409 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:40.409 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.409 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:40.409 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:40.409 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:40.409 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:40.409 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:40.409 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.409 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:40.409 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:40.409 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:40.409 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:40.410 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:40.410 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.410 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:40.410 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:40.410 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:40.410 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:40.410 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.410 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:40.410 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:40.410 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:40.410 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:40.410 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.411 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:40.411 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:40.411 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:40.411 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:06:40.411 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: osd.0 [v2:172.21.15.116:6800/3889754334,v1:172.21.15.116:6801/3889754334] boot 2024-06-01T02:06:40.411 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: osdmap e121: 8 total, 8 up, 8 in 2024-06-01T02:06:40.411 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T02:06:40.411 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:40.411 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:40.411 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:40.412 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:40.412 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.412 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:40.412 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.412 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.412 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.412 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:40.412 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:40.412 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:40.412 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:40.412 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.412 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:40.412 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:40.412 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:40.412 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:40.413 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:40.413 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.413 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:40.413 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:40.413 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:40.413 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:40.413 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:40.413 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.413 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:40.413 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:40.413 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:40.413 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:40.414 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:40.414 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.414 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:40.414 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:40.414 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:40.414 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:40.414 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:40.414 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.414 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:40.414 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:40.414 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:40.415 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:40.415 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.415 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:40.415 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:40.415 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:40.415 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:40.415 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:40.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:40.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:40.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:06:40.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: osd.0 [v2:172.21.15.116:6800/3889754334,v1:172.21.15.116:6801/3889754334] boot 2024-06-01T02:06:40.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: osdmap e121: 8 total, 8 up, 8 in 2024-06-01T02:06:40.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T02:06:40.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:40.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:40.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:40.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:40.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:40.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:40.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:40.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:40.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:40.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:40.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:40.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:40.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:40.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:40.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:40.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:40.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:40.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:40.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:40.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:40.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:40.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:40.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:40.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:40.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.701 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:40.701 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:40.701 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:40.701 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:40.701 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:40.701 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.701 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:40.701 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:40.702 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:40.702 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:40.702 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:40.702 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:40.702 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:40.702 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:40.703 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:40.703 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:41.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:41.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:41.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:41.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: osdmap e122: 8 total, 8 up, 8 in 2024-06-01T02:06:41.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:41.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:41.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:41.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:41.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:41.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:41.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:41.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:41.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:41.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:41.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:41.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:41.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:41.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:41.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:41.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:41.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:41.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:41.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:41.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:41.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:41.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:41.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:41.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:41.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:41.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:41.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:41.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:41.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:41.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: osdmap e122: 8 total, 8 up, 8 in 2024-06-01T02:06:41.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:41.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:41.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:41.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:41.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:41.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:41.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:41.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:41.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:41.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:41.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:41.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:41.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:41.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:41.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:41.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:41.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:41.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:41.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:41.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:41.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:41.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:41.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:41.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:41.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:41.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:41.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:41.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:41.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:41.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:41.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: pgmap v76: 161 pgs: 7 active+clean+remapped, 21 active+undersized+degraded, 35 active+undersized, 98 active+clean; 585 KiB data, 117 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 76/726 objects degraded (10.468%); 9/726 objects misplaced (1.240%); 0 B/s, 0 objects/s recovering 2024-06-01T02:06:41.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:41.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:41.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:41.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:41.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:41.553 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:41.554 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:41.554 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:41.554 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:41.554 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:41.554 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:41.554 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:41.554 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:41.555 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:41.555 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:41.555 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:41.555 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:41.555 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:41.555 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:41.555 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:41.555 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:41.555 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:41.555 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:41.555 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:41.555 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:41.556 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:41.556 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:41.556 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:41.556 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:41.556 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: pgmap v76: 161 pgs: 7 active+clean+remapped, 21 active+undersized+degraded, 35 active+undersized, 98 active+clean; 585 KiB data, 117 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 76/726 objects degraded (10.468%); 9/726 objects misplaced (1.240%); 0 B/s, 0 objects/s recovering 2024-06-01T02:06:41.556 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:41.556 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:41.556 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:41.556 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:41.556 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:41.556 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:41.557 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:41.557 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:41.557 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:41.557 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:41.557 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:41.557 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:41.557 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:41.557 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:41.557 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:41.557 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:41.557 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:41.557 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:41.557 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:41.557 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:41.558 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:41.558 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:41.558 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:41.558 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:41.558 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:41.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:41.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:41.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:41.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: osdmap e122: 8 total, 8 up, 8 in 2024-06-01T02:06:41.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:41.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:41.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:41.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:41.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:41.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:41.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:41.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:41.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:41.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:41.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:41.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:41.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:41.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:41.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:41.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:41.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:41.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:41.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:41.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:41.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:41.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:41.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:41.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:41.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:41.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:41.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:41.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:41.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:41.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:41.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: pgmap v76: 161 pgs: 7 active+clean+remapped, 21 active+undersized+degraded, 35 active+undersized, 98 active+clean; 585 KiB data, 117 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 76/726 objects degraded (10.468%); 9/726 objects misplaced (1.240%); 0 B/s, 0 objects/s recovering 2024-06-01T02:06:41.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:41.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:41.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:41.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:41.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:41.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:41.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:41.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:41.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:41.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:41.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:41.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:41.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:41.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:41.701 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:41.701 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:41.701 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:41.701 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:41.701 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:41.702 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:41.702 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:41.702 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:41.702 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:41.702 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:41.702 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:42.483 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:42.485 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:42.485 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:42.485 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:42.485 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:42.486 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: osdmap e123: 8 total, 8 up, 8 in 2024-06-01T02:06:42.486 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:42.486 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:42.486 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:42.486 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:42.486 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:42.487 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:42.487 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:42.487 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:42.487 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:42.487 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:42.487 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:42.487 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:42.488 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:42.488 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:42.488 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:42.488 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:42.488 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:42.488 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:42.489 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:42.489 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:42.489 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:42.489 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:42.489 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:42.489 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:42.490 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:42.490 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:42.491 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:42.491 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:42.491 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:42.491 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: osdmap e123: 8 total, 8 up, 8 in 2024-06-01T02:06:42.491 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:42.491 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:42.491 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:42.492 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:42.492 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:42.492 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:42.492 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:42.492 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:42.492 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:42.492 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:42.493 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:42.493 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:42.494 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:42.494 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:42.494 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:42.494 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:42.494 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:42.494 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:42.494 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:42.494 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:42.494 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:42.494 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:42.494 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:42.494 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:42.495 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:42.495 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:42.495 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:42.495 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:42.495 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:42.495 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:42.495 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:06:42.495 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:42.496 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:42.496 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:42.496 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:42.496 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:42.496 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:42.496 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:42.496 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:42.496 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:42.496 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:42.496 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:42.496 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:42.497 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:42.497 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:42.497 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:42.497 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:42.497 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:42.497 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:42.497 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:42.497 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:42.497 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:42.497 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:06:42.497 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:42.497 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:42.498 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:42.498 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:42.498 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:42.498 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:42.498 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:42.498 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:42.498 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:42.498 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:42.498 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:42.498 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:42.498 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:42.499 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:42.499 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: osdmap e123: 8 total, 8 up, 8 in 2024-06-01T02:06:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:42.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:42.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:42.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:42.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:42.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:06:42.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:42.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:42.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:42.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:42.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:42.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:42.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:42.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:42.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:42.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:42.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:42.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:42.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:42.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:42.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:43.508 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:43.508 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:43.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:43.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:43.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:43.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:43.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:43.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:43.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:43.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:43.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:43.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:43.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:43.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:43.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:43.510 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:43.510 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:43.510 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:43.510 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:43.510 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:43.510 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:43.510 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:43.510 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:43.510 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:43.510 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:43.510 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:43.510 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:43.510 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:43.511 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:43.511 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:43.511 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:43.511 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:43.511 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:43.511 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:43.511 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:43.511 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:43.511 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:43.512 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:43.512 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:43.512 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:43.512 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:43.512 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:43.512 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:43.512 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:43.512 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:43.512 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:43.512 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:43.512 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:43.512 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:43.512 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:43.512 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:43.513 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:43.513 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:43.513 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:43.513 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:43.513 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:43.513 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:43.513 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:43.514 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:43.514 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:43.514 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:43.514 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:43.514 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: pgmap v78: 161 pgs: 2 active+clean+remapped, 7 active+undersized+degraded, 13 active+undersized, 139 active+clean; 585 KiB data, 122 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 32/726 objects degraded (4.408%); 24 B/s, 1 objects/s recovering 2024-06-01T02:06:43.514 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:43.517 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:43.517 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:43.517 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:43.517 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:43.517 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:43.517 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:43.517 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:43.517 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:43.518 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:43.518 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:43.518 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:43.518 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:43.518 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:43.518 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:43.518 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:43.518 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:43.518 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:43.518 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:43.518 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:43.518 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:43.518 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:43.519 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:43.519 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:43.519 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:43.519 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:43.519 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:43.519 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:43.519 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:43.519 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:43.519 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: pgmap v78: 161 pgs: 2 active+clean+remapped, 7 active+undersized+degraded, 13 active+undersized, 139 active+clean; 585 KiB data, 122 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 32/726 objects degraded (4.408%); 24 B/s, 1 objects/s recovering 2024-06-01T02:06:43.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:43.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:43.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:43.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:43.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:43.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:43.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:43.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:43.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:43.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:43.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:43.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:43.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:43.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:43.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:43.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:43.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:43.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:43.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:43.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:43.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:43.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:43.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:43.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:43.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:43.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:43.694 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:06:43.504Z caller=manager.go:609 level=warn component="rule manager" group=nodes msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on(instance) group_left(nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi116\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:06:43.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:43.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:43.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:43.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:43.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:43.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:43.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:43.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:43.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:43.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:43.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:43.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:43.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:43.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:43.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:43.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:43.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:43.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:43.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:43.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:43.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:43.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:43.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:43.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:43.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:43.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:43.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:43.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:43.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:43.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:43.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:43.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:43.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:43.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: pgmap v78: 161 pgs: 2 active+clean+remapped, 7 active+undersized+degraded, 13 active+undersized, 139 active+clean; 585 KiB data, 122 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 32/726 objects degraded (4.408%); 24 B/s, 1 objects/s recovering 2024-06-01T02:06:43.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:43.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:43.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:43.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:43.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:43.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:43.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:43.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:43.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:43.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:43.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:43.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:43.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:43.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:43.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:43.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:43.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:43.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:43.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:43.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:43.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:43.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:43.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:43.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:43.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:43.700 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:44.693 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:44.693 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:44.693 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:44.693 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:44.693 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:44.694 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:44.694 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:44.694 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:44.694 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:44.694 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:44.694 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:44.694 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:44.694 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:44.694 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:44.694 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:44.694 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:44.694 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:44.695 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:44.695 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:44.695 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:44.695 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:44.695 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:44.695 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:44.695 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:44.695 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:44.695 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:44.695 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:44.695 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:44.695 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: Health check update: Degraded data redundancy: 32/726 objects degraded (4.408%), 7 pgs degraded (PG_DEGRADED) 2024-06-01T02:06:44.696 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:44.696 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:44.696 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:44.696 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:44.696 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:44.696 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:44.696 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:44.696 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:44.697 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:44.697 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:44.697 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:44.697 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:44.697 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:44.697 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:44.697 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:44.697 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:44.697 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:44.697 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:44.697 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:44.697 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:44.697 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:44.698 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:44.698 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:44.698 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:44.698 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:44.698 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:44.698 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:44.698 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:44.698 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: Health check update: Degraded data redundancy: 32/726 objects degraded (4.408%), 7 pgs degraded (PG_DEGRADED) 2024-06-01T02:06:44.698 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:44.698 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:44.698 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:44.698 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:44.698 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:44.699 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:44.699 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:44.699 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:44.699 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:44.699 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:44.699 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:44.699 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:44.699 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:44.699 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:44.699 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:44.699 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:44.699 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:44.699 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:44.700 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:44.700 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:44.700 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:44.700 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:44.700 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:44.700 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:44.700 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:44.700 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:44.700 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:44.700 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:44.700 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:44.700 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:44.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:44.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:44.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:44.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:44.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:44.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:44.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:44.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:44.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:44.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:44.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:44.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:44.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:44.701 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:44.702 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:44.702 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:44.702 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:44.702 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:44.702 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:44.702 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:44.702 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:44.702 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:44.702 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:44.702 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:44.703 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:44.703 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:44.703 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:44.703 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:44.703 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:44.703 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:44.703 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:44.703 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:44.703 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:44.703 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:44.703 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:44.704 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:44.704 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:44.704 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:44.704 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:44.704 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:44.704 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:44.704 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:44.704 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:44.704 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:44.704 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:44.704 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:44.704 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:44.704 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:44.704 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:44.704 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:44.705 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:44.705 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:44.705 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: Health check update: Degraded data redundancy: 32/726 objects degraded (4.408%), 7 pgs degraded (PG_DEGRADED) 2024-06-01T02:06:44.705 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:44.705 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:44.705 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:44.705 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:44.705 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:44.705 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:44.705 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:44.705 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:44.705 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:44.705 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:44.705 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:44.706 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:44.706 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:44.706 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:44.706 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:44.706 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:44.706 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:44.706 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:44.706 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:44.706 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:44.706 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:44.706 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:44.706 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:44.706 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:44.707 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:44.707 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:44.707 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.673 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:45.673 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:45.673 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:45.674 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.674 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.674 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.674 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.674 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.674 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:45.674 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.674 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.674 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.674 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.674 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.674 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:45.674 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.675 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.675 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.675 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.675 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.675 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:45.675 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.675 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.675 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.675 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.675 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.675 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:45.675 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.675 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.676 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:45.676 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:45.676 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:45.676 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.676 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.676 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.676 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.676 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.676 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:45.676 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.677 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.677 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.677 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.677 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.677 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:45.677 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.677 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.677 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.677 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.677 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.677 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:45.677 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.677 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.677 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.677 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.678 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.678 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:45.678 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.678 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.678 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.678 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.678 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.678 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:45.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: pgmap v79: 161 pgs: 161 active+clean; 585 KiB data, 122 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 18 B/s, 1 objects/s recovering 2024-06-01T02:06:45.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:45.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.679 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.680 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.680 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.680 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.680 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.680 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.680 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.680 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.680 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.680 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.680 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.680 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.680 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.680 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.680 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.681 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.681 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.681 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.682 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.682 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.682 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.682 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:45.682 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: pgmap v79: 161 pgs: 161 active+clean; 585 KiB data, 122 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 18 B/s, 1 objects/s recovering 2024-06-01T02:06:45.682 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.682 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.682 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.682 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.682 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.683 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:45.683 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.683 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.683 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.683 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.683 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.683 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.683 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.683 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.683 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.683 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.683 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.683 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.684 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.684 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.684 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.684 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.684 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.684 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.684 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.684 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.684 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.684 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.684 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.684 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.684 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:45.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:45.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:45.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:45.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:45.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:45.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:45.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:45.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: pgmap v79: 161 pgs: 161 active+clean; 585 KiB data, 122 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 18 B/s, 1 objects/s recovering 2024-06-01T02:06:45.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:45.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:45.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:45.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:45.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:45.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:45.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:46.657 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:46.657 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:46.657 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:46.657 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:46.657 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:46.657 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:46.657 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:46.657 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:46.657 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:46.657 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:46.658 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:46.658 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:46.658 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:46.658 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:46.658 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:46.658 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:46.658 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:46.658 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:46.658 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:46.658 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:46.658 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:46.658 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:46.658 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:46.658 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 32/726 objects degraded (4.408%), 7 pgs degraded) 2024-06-01T02:06:46.659 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: Cluster is now healthy 2024-06-01T02:06:46.659 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:46.659 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:06:46.659 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:46.659 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:46.659 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:46.659 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:46.659 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:46.660 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:46.660 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:46.660 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:46.660 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:46.660 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:46.660 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:46.660 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:46.660 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:46.660 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:46.660 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:46.660 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:46.660 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:46.660 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:46.660 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:46.661 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:46.661 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:46.661 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:46.661 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:46.661 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:46.661 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:46.661 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:46.661 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 32/726 objects degraded (4.408%), 7 pgs degraded) 2024-06-01T02:06:46.661 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: Cluster is now healthy 2024-06-01T02:06:46.661 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:46.661 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:06:46.661 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:46.661 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:46.661 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:46.662 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:46.662 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:46.662 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:46.662 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:46.662 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:46.662 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:46.662 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:46.662 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:46.662 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:46.662 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:46.662 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:46.662 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:46.662 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:46.662 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:46.663 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:46.663 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:46.663 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:46.663 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:46.663 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:46.663 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:46.663 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:46.663 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:46.663 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:46.663 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:46.663 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:46.663 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:46.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:46.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:46.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:46.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:46.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:46.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:46.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:46.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:46.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:46.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:46.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:46.664 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:46.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:46.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:46.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:46.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:46.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:46.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:46.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:46.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:46.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:46.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:46.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:46.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:46.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:46.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:46.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:46.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:46.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:46.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:46.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:46.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:46.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:46.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:46.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:46.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 32/726 objects degraded (4.408%), 7 pgs degraded) 2024-06-01T02:06:46.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: Cluster is now healthy 2024-06-01T02:06:46.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:46.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:06:46.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:46.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:46.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:46.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:46.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:46.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:46.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:46.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:46.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:46.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:46.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:46.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:46.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:46.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:46.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:46.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:46.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:46.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:46.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:46.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:46.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:46.699 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:47.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:06:46 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:06:46] "GET /metrics HTTP/1.1" 200 38298 "" "Prometheus/2.33.4" 2024-06-01T02:06:47.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:47.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:47.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:47.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:47.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:47.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:47.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:47.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:47.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:47.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:47.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:47.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:47.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:47.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:47.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:47.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:47.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:47.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:47.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:47.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:47.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:47.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: pgmap v80: 161 pgs: 161 active+clean; 585 KiB data, 122 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 14 B/s, 1 objects/s recovering 2024-06-01T02:06:47.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:47.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:47.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:47.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:47.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:47.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:47.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:47.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:47.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:47.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:47.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:47.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:47.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:47.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:47.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:47.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:47.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:47.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:47.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:47.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:47.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:47.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:47.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:47.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:47.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:47.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:47.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:47.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:47.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:47.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:47.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:47.757 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:47.757 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:47.757 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:47.757 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:47.757 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:47.758 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:47.758 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:47.758 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:47.758 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:47.758 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:47.758 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:47.758 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:47.758 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:47.758 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:47.758 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:47.758 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:47.758 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:47.758 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:47.759 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:47.759 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:47.759 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:47.759 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: pgmap v80: 161 pgs: 161 active+clean; 585 KiB data, 122 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 14 B/s, 1 objects/s recovering 2024-06-01T02:06:47.759 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:47.759 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:47.759 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:47.759 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:47.759 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:47.759 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:47.759 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:47.760 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:47.760 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:47.760 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:47.760 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:47.760 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:47.760 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:47.760 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:47.760 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:47.760 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:47.760 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:47.760 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:47.760 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:47.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:47.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:47.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:47.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:47.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:47.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:47.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:47.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:47.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:47.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: pgmap v80: 161 pgs: 161 active+clean; 585 KiB data, 122 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 14 B/s, 1 objects/s recovering 2024-06-01T02:06:47.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:47.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:47.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:47.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:47.762 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:47.762 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:47.762 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:47.762 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:47.762 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:47.762 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:47.762 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:47.762 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:47.762 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:47.762 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:47.763 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:47.763 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:47.763 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:47.763 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:47.763 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:47.763 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:47.763 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:47.763 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:47.763 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:47.763 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:47.763 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:47.763 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:47.763 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:47.763 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:47.764 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:47.764 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:47.764 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:47.764 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:47.764 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:47.764 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:47.764 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:47.764 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:47.764 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:47.764 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:47.764 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:47.764 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:47.764 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:47.765 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:47.765 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:47.765 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:47.765 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:47.765 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:47.765 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:47.765 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:47.765 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:47.765 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:47.765 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:47.765 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:47.765 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:47.765 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:47.765 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:48.642 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:48.642 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:48.642 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:48.642 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:48.642 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:48.642 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:48.642 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:48.643 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:48.643 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:48.643 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:48.643 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:48.643 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:48.643 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:48.643 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:48.643 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:48.643 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:48.643 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:48.643 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:48.643 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:48.643 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:48.644 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:48.644 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:48.644 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:48.644 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:48.644 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:48.644 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:48.644 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:48.644 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:48.644 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:48.644 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:48.644 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:48.644 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:48.644 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:48.644 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:48.644 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:48.645 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:48.645 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:48.645 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:48.645 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:48.645 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:48.645 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:48.645 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:48.645 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:48.645 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:48.645 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:48.645 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:48.645 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:48.645 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:48.646 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:48.646 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:48.646 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:48.646 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:48.646 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-06-01T02:06:48.646 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:48.646 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:48.646 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:48.647 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:48.647 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:48.647 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:48.647 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:48.647 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:48.647 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:48.647 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:48.647 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:48.647 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:48.647 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:48.647 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:48.647 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:48.647 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:48.648 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:48.648 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:48.648 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:48.648 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:48.648 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:48.648 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:48.648 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:48.648 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:48.648 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:48.648 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:48.648 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:48.648 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:48.648 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:48.649 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:48.649 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:48.649 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:48.649 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:48.649 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:48.649 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:48.649 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:48.649 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:48.649 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:48.649 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:48.649 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:48.649 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:48.649 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:48.649 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:48.650 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:48.650 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:48.650 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:48.650 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:48.650 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:48.650 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:48.650 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:48.650 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:48.650 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:48.650 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-06-01T02:06:48.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:48.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:48.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:48.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:48.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:48.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:48.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:48.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:48.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:48.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:48.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:48.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:48.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:48.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:48.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:48.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:48.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:48.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:48.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:48.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:48.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:48.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:48.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:48.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:48.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:48.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:48.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:48.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:48.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:48.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:48.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:48.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:48.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:48.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:48.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:48.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:48.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:48.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:48.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:48.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:48.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:48.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:48.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:48.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:06:48.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:48.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:48.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:48.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:48.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:06:48.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:48.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:48.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:48.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-06-01T02:06:48.944 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:06:48 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:06:48.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"smithi161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"172.21.15.161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:06:49.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:49 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:49.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:49 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:49.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:49 smithi116 ceph-mon[100606]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-06-01T02:06:49.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:49 smithi116 ceph-mon[100606]: Upgrade: osd.0 is safe to restart 2024-06-01T02:06:49.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:49 smithi116 ceph-mon[100606]: pgmap v81: 161 pgs: 161 active+clean; 585 KiB data, 122 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13 B/s, 0 objects/s recovering 2024-06-01T02:06:49.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:49.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:49.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:49.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:49 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:49.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:49 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:49.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:49 smithi116 ceph-mon[89502]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-06-01T02:06:49.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:49 smithi116 ceph-mon[89502]: Upgrade: osd.0 is safe to restart 2024-06-01T02:06:49.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:49 smithi116 ceph-mon[89502]: pgmap v81: 161 pgs: 161 active+clean; 585 KiB data, 122 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13 B/s, 0 objects/s recovering 2024-06-01T02:06:49.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:49.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:49.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:49.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:49 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:49.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:49 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:06:49.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:49 smithi161 ceph-mon[84406]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-06-01T02:06:49.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:49 smithi161 ceph-mon[84406]: Upgrade: osd.0 is safe to restart 2024-06-01T02:06:49.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:49 smithi161 ceph-mon[84406]: pgmap v81: 161 pgs: 161 active+clean; 585 KiB data, 122 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13 B/s, 0 objects/s recovering 2024-06-01T02:06:49.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:49.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:49.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:50.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:50.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:50.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:51.020 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:51.021 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:51.021 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:51.021 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:51.021 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:51.021 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:06:51.741 INFO:teuthology.orchestra.run.smithi116.stdout:true 2024-06-01T02:06:51.927 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:51 smithi116 ceph-mon[100606]: pgmap v82: 161 pgs: 161 active+clean; 585 KiB data, 122 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 10 B/s, 0 objects/s recovering 2024-06-01T02:06:51.927 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:51 smithi116 ceph-mon[89502]: pgmap v82: 161 pgs: 161 active+clean; 585 KiB data, 122 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 10 B/s, 0 objects/s recovering 2024-06-01T02:06:51.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:51 smithi161 ceph-mon[84406]: pgmap v82: 161 pgs: 161 active+clean; 585 KiB data, 122 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 10 B/s, 0 objects/s recovering 2024-06-01T02:06:52.424 INFO:teuthology.orchestra.run.smithi116.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-01T02:06:52.424 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi116 smithi116 running 2s ago 15m - - 2024-06-01T02:06:52.424 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi161 smithi161 running 12s ago 15m - - 2024-06-01T02:06:52.424 INFO:teuthology.orchestra.run.smithi116.stdout:alertmanager.a smithi116 *:9093,9094 running (4m) 2s ago 9m 21.2M - 0.23.0 ba2b418f427c e0c297162ded 2024-06-01T02:06:52.424 INFO:teuthology.orchestra.run.smithi116.stdout:grafana.a smithi161 *:3000 running (4m) 12s ago 9m 52.1M - 8.3.5 dad864ee21e9 00b0a78ee88e 2024-06-01T02:06:52.424 INFO:teuthology.orchestra.run.smithi116.stdout:iscsi.foo.smithi116.gjuxzb smithi116 running (3m) 2s ago 8m 51.1M - 3.5 e1d6a67b021e 0e191fe3e3bb 2024-06-01T02:06:52.424 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.x smithi161 *:8443,9283,8765 running (3m) 12s ago 14m 515M - 19.0.0-2946-g3d89accf b045ac545f82 2d7fbb85e5f7 2024-06-01T02:06:52.424 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.y smithi116 *:8443,9283,8765 running (2m) 2s ago 16m 453M - 19.0.0-2946-g3d89accf b045ac545f82 bc9627c27a57 2024-06-01T02:06:52.425 INFO:teuthology.orchestra.run.smithi116.stdout:mon.a smithi116 running (2m) 2s ago 16m 55.7M 2048M 19.0.0-2946-g3d89accf b045ac545f82 b5e9b6f873ff 2024-06-01T02:06:52.425 INFO:teuthology.orchestra.run.smithi116.stdout:mon.b smithi161 running (61s) 12s ago 14m 33.3M 2048M 19.0.0-2946-g3d89accf b045ac545f82 aaa08c03cfc8 2024-06-01T02:06:52.425 INFO:teuthology.orchestra.run.smithi116.stdout:mon.c smithi116 running (77s) 2s ago 15m 42.2M 2048M 19.0.0-2946-g3d89accf b045ac545f82 bd0fb0e603fa 2024-06-01T02:06:52.425 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.a smithi116 *:9100 running (5m) 2s ago 10m 21.7M - 1.3.1 1dbe0e931976 e347748d17bb 2024-06-01T02:06:52.425 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.b smithi161 *:9100 running (4m) 12s ago 10m 17.4M - 1.3.1 1dbe0e931976 6e66182f5a1a 2024-06-01T02:06:52.425 INFO:teuthology.orchestra.run.smithi116.stdout:osd.0 smithi116 running (18s) 2s ago 14m 42.9M 4096M 19.0.0-2946-g3d89accf b045ac545f82 fd69ed064e41 2024-06-01T02:06:52.425 INFO:teuthology.orchestra.run.smithi116.stdout:osd.1 smithi116 running (13m) 2s ago 13m 58.3M 4096M 17.2.0 e1d6a67b021e 6ff7fc5abe58 2024-06-01T02:06:52.425 INFO:teuthology.orchestra.run.smithi116.stdout:osd.2 smithi116 running (13m) 2s ago 13m 55.6M 4096M 17.2.0 e1d6a67b021e 6aecbb10256b 2024-06-01T02:06:52.425 INFO:teuthology.orchestra.run.smithi116.stdout:osd.3 smithi116 running (13m) 2s ago 13m 56.5M 4096M 17.2.0 e1d6a67b021e a802290f5290 2024-06-01T02:06:52.425 INFO:teuthology.orchestra.run.smithi116.stdout:osd.4 smithi161 running (12m) 12s ago 12m 58.9M 4096M 17.2.0 e1d6a67b021e 70b3f89b0bd9 2024-06-01T02:06:52.425 INFO:teuthology.orchestra.run.smithi116.stdout:osd.5 smithi161 running (12m) 12s ago 12m 53.2M 4096M 17.2.0 e1d6a67b021e 68358b78a25d 2024-06-01T02:06:52.425 INFO:teuthology.orchestra.run.smithi116.stdout:osd.6 smithi161 running (11m) 12s ago 11m 54.1M 4096M 17.2.0 e1d6a67b021e 50abeb0551e7 2024-06-01T02:06:52.426 INFO:teuthology.orchestra.run.smithi116.stdout:osd.7 smithi161 running (11m) 12s ago 11m 56.8M 4096M 17.2.0 e1d6a67b021e 6004e6983735 2024-06-01T02:06:52.426 INFO:teuthology.orchestra.run.smithi116.stdout:prometheus.a smithi161 *:9095 running (3m) 12s ago 10m 74.7M - 2.33.4 514e6a882f6e e977f6ca1d73 2024-06-01T02:06:52.426 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi116.vzjrxj smithi116 *:8000 running (9m) 2s ago 9m 76.3M - 17.2.0 e1d6a67b021e df4fe3c1dbc3 2024-06-01T02:06:52.426 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi161.ashqxz smithi161 *:8000 running (8m) 12s ago 8m 77.2M - 17.2.0 e1d6a67b021e 3eecc1f3e2d3 2024-06-01T02:06:52.426 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi116.hokbmf smithi116 *:80 running (8m) 2s ago 8m 76.7M - 17.2.0 e1d6a67b021e 10950ed75dbd 2024-06-01T02:06:52.426 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi161.axjlzw smithi161 *:80 running (8m) 12s ago 8m 75.9M - 17.2.0 e1d6a67b021e 0b5d5ce47772 2024-06-01T02:06:52.828 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:52 smithi116 ceph-mon[100606]: from='client.44214 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:06:52.828 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:52 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:06:52.828 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:52 smithi116 ceph-mon[89502]: from='client.44214 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:06:52.828 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:52 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:06:52.828 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:06:52.829 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": { 2024-06-01T02:06:52.829 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 3 2024-06-01T02:06:52.829 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:06:52.829 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": { 2024-06-01T02:06:52.829 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-01T02:06:52.829 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:06:52.829 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": { 2024-06-01T02:06:52.829 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 7, 2024-06-01T02:06:52.829 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 1 2024-06-01T02:06:52.829 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:06:52.829 INFO:teuthology.orchestra.run.smithi116.stdout: "rgw": { 2024-06-01T02:06:52.830 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-06-01T02:06:52.830 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:06:52.830 INFO:teuthology.orchestra.run.smithi116.stdout: "overall": { 2024-06-01T02:06:52.830 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 11, 2024-06-01T02:06:52.830 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 6 2024-06-01T02:06:52.830 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T02:06:52.830 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:06:52.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:52 smithi161 ceph-mon[84406]: from='client.44214 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:06:52.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:52 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:06:53.168 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:06:53.168 INFO:teuthology.orchestra.run.smithi116.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-01T02:06:53.168 INFO:teuthology.orchestra.run.smithi116.stdout: "in_progress": true, 2024-06-01T02:06:53.168 INFO:teuthology.orchestra.run.smithi116.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-01T02:06:53.168 INFO:teuthology.orchestra.run.smithi116.stdout: "services_complete": [ 2024-06-01T02:06:53.168 INFO:teuthology.orchestra.run.smithi116.stdout: "mon", 2024-06-01T02:06:53.168 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr" 2024-06-01T02:06:53.168 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-06-01T02:06:53.168 INFO:teuthology.orchestra.run.smithi116.stdout: "progress": "6/23 daemons upgraded", 2024-06-01T02:06:53.169 INFO:teuthology.orchestra.run.smithi116.stdout: "message": "Currently upgrading osd daemons", 2024-06-01T02:06:53.169 INFO:teuthology.orchestra.run.smithi116.stdout: "is_paused": false 2024-06-01T02:06:53.169 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:06:53.569 INFO:teuthology.orchestra.run.smithi116.stdout:HEALTH_OK 2024-06-01T02:06:53.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:53 smithi116 ceph-mon[89502]: from='client.64186 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:06:53.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:53 smithi116 ceph-mon[89502]: from='client.64192 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:06:53.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:53 smithi116 ceph-mon[89502]: pgmap v83: 161 pgs: 161 active+clean; 585 KiB data, 122 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 9 B/s, 0 objects/s recovering 2024-06-01T02:06:53.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:53 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/3239233451' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:53.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-01T02:06:53.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:53.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:53 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/3288360476' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:06:53.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:53 smithi116 ceph-mon[100606]: from='client.64186 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:06:53.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:53 smithi116 ceph-mon[100606]: from='client.64192 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:06:53.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:53 smithi116 ceph-mon[100606]: pgmap v83: 161 pgs: 161 active+clean; 585 KiB data, 122 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 9 B/s, 0 objects/s recovering 2024-06-01T02:06:53.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:53 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/3239233451' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:53.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-01T02:06:53.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:53.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:53 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/3288360476' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:06:53.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:53 smithi161 ceph-mon[84406]: from='client.64186 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:06:53.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:53 smithi161 ceph-mon[84406]: from='client.64192 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:06:53.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:53 smithi161 ceph-mon[84406]: pgmap v83: 161 pgs: 161 active+clean; 585 KiB data, 122 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 9 B/s, 0 objects/s recovering 2024-06-01T02:06:53.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:53 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/3239233451' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:06:53.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-01T02:06:53.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:06:53.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:53 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/3288360476' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:06:53.846 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:06:53 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:06:53.505Z caller=manager.go:609 level=warn component="rule manager" group=nodes msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on(instance) group_left(nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi116\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:06:54.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:54 smithi161 ceph-mon[84406]: from='client.74156 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:06:54.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:54 smithi161 ceph-mon[84406]: Upgrade: Updating osd.0 2024-06-01T02:06:54.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:54 smithi161 ceph-mon[84406]: Deploying daemon osd.0 on smithi116 2024-06-01T02:06:55.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:54 smithi116 ceph-mon[89502]: from='client.74156 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:06:55.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:54 smithi116 ceph-mon[89502]: Upgrade: Updating osd.0 2024-06-01T02:06:55.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:54 smithi116 ceph-mon[89502]: Deploying daemon osd.0 on smithi116 2024-06-01T02:06:55.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:54 smithi116 ceph-mon[100606]: from='client.74156 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:06:55.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:54 smithi116 ceph-mon[100606]: Upgrade: Updating osd.0 2024-06-01T02:06:55.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:54 smithi116 ceph-mon[100606]: Deploying daemon osd.0 on smithi116 2024-06-01T02:06:55.688 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:55 smithi116 ceph-mon[89502]: pgmap v84: 161 pgs: 161 active+clean; 585 KiB data, 122 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:06:55.689 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:55 smithi116 ceph-mon[100606]: pgmap v84: 161 pgs: 161 active+clean; 585 KiB data, 122 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:06:55.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:55 smithi161 ceph-mon[84406]: pgmap v84: 161 pgs: 161 active+clean; 585 KiB data, 122 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:06:57.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:06:56 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:06:56] "GET /metrics HTTP/1.1" 200 38298 "" "Prometheus/2.33.4" 2024-06-01T02:06:57.293 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:57 smithi116 systemd[1]: Stopping Ceph osd.0 for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:06:57.793 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:57 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0[107294]: 2024-06-01T02:06:57.335+0000 7f6b33e02700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-01T02:06:57.793 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:57 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0[107294]: 2024-06-01T02:06:57.335+0000 7f6b33e02700 -1 osd.0 123 *** Got signal Terminated *** 2024-06-01T02:06:57.793 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:57 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0[107294]: 2024-06-01T02:06:57.335+0000 7f6b33e02700 -1 osd.0 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-01T02:06:58.180 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:57 smithi116 ceph-mon[89502]: pgmap v85: 161 pgs: 161 active+clean; 585 KiB data, 122 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:06:58.180 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:57 smithi116 ceph-mon[89502]: osd.0 marked itself down and dead 2024-06-01T02:06:58.181 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:57 smithi116 ceph-mon[100606]: pgmap v85: 161 pgs: 161 active+clean; 585 KiB data, 122 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:06:58.181 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:57 smithi116 ceph-mon[100606]: osd.0 marked itself down and dead 2024-06-01T02:06:58.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:57 smithi161 ceph-mon[84406]: pgmap v85: 161 pgs: 161 active+clean; 585 KiB data, 122 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:06:58.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:57 smithi161 ceph-mon[84406]: osd.0 marked itself down and dead 2024-06-01T02:06:58.457 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:58 smithi116 podman[113586]: 2024-06-01 02:06:58.185079874 +0000 UTC m=+0.965506371 container died fd69ed064e41e01ff2870debc06157ed3bd26b9c1a067eebf838e446a24f44c1 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-06-01T02:06:58.793 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:58 smithi116 podman[113586]: 2024-06-01 02:06:58.669774531 +0000 UTC m=+1.450201028 container cleanup fd69ed064e41e01ff2870debc06157ed3bd26b9c1a067eebf838e446a24f44c1 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=) 2024-06-01T02:06:58.793 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:58 smithi116 bash[113586]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0 2024-06-01T02:06:58.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:58 smithi161 ceph-mon[84406]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:06:58.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:58 smithi161 ceph-mon[84406]: osdmap e124: 8 total, 7 up, 8 in 2024-06-01T02:06:58.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:58 smithi161 ceph-mon[84406]: osdmap e125: 8 total, 7 up, 8 in 2024-06-01T02:06:58.945 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:06:58 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:06:58.660Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"smithi161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.116\", device_class=\"ssd\", hostname=\"smithi116\", instance=\"172.21.15.161:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.116\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:06:59.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:58 smithi116 ceph-mon[89502]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:06:59.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:58 smithi116 ceph-mon[89502]: osdmap e124: 8 total, 7 up, 8 in 2024-06-01T02:06:59.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:58 smithi116 ceph-mon[89502]: osdmap e125: 8 total, 7 up, 8 in 2024-06-01T02:06:59.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:58 smithi116 ceph-mon[100606]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:06:59.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:58 smithi116 ceph-mon[100606]: osdmap e124: 8 total, 7 up, 8 in 2024-06-01T02:06:59.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:58 smithi116 ceph-mon[100606]: osdmap e125: 8 total, 7 up, 8 in 2024-06-01T02:06:59.294 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:58 smithi116 podman[113603]: 2024-06-01 02:06:58.988093168 +0000 UTC m=+0.802496526 container remove fd69ed064e41e01ff2870debc06157ed3bd26b9c1a067eebf838e446a24f44c1 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_CLEAN=True) 2024-06-01T02:06:59.620 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:59 smithi116 podman[113690]: 2024-06-01 02:06:59.251054738 +0000 UTC m=+0.017652788 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:06:59.620 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:59 smithi116 podman[113690]: 2024-06-01 02:06:59.414499212 +0000 UTC m=+0.181097251 container create 30b2391492464bdc5f274e1eaf4389c0ac562d933c51a0743b5299b07bc71500 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, ceph=True) 2024-06-01T02:06:59.620 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:59 smithi116 podman[113690]: 2024-06-01 02:06:59.625191458 +0000 UTC m=+0.391789489 container init 30b2391492464bdc5f274e1eaf4389c0ac562d933c51a0743b5299b07bc71500 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-06-01T02:06:59.879 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:59 smithi116 podman[113690]: 2024-06-01 02:06:59.628822572 +0000 UTC m=+0.395420623 container start 30b2391492464bdc5f274e1eaf4389c0ac562d933c51a0743b5299b07bc71500 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-deactivate, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-01T02:06:59.879 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:59 smithi116 podman[113690]: 2024-06-01 02:06:59.750444508 +0000 UTC m=+0.517042555 container attach 30b2391492464bdc5f274e1eaf4389c0ac562d933c51a0743b5299b07bc71500 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-01T02:07:00.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:59 smithi161 ceph-mon[84406]: pgmap v87: 161 pgs: 22 stale+active+clean, 139 active+clean; 585 KiB data, 122 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:07:00.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:59 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:00.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:59 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:00.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:06:59 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:00.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:59 smithi116 ceph-mon[89502]: pgmap v87: 161 pgs: 22 stale+active+clean, 139 active+clean; 585 KiB data, 122 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:07:00.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:00.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:00.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:06:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:00.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:59 smithi116 ceph-mon[100606]: pgmap v87: 161 pgs: 22 stale+active+clean, 139 active+clean; 585 KiB data, 122 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:07:00.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:59 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:00.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:59 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:00.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:06:59 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:00.294 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:06:59 smithi116 podman[113690]: 2024-06-01 02:06:59.917135222 +0000 UTC m=+0.683733271 container died 30b2391492464bdc5f274e1eaf4389c0ac562d933c51a0743b5299b07bc71500 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-deactivate, ceph=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528) 2024-06-01T02:07:00.683 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:00 smithi116 podman[113716]: 2024-06-01 02:07:00.553887715 +0000 UTC m=+0.626877864 container remove 30b2391492464bdc5f274e1eaf4389c0ac562d933c51a0743b5299b07bc71500 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-06-01T02:07:00.683 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:00 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.0.service: Deactivated successfully. 2024-06-01T02:07:00.683 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:00 smithi116 systemd[1]: Stopped Ceph osd.0 for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:07:00.683 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:00 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.0.service: Consumed 1.901s CPU time. 2024-06-01T02:07:00.999 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:00 smithi116 systemd[1]: Starting Ceph osd.0 for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:07:01.270 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:01 smithi116 podman[113817]: 2024-06-01 02:07:00.912069848 +0000 UTC m=+0.017373002 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:07:01.270 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:01 smithi116 podman[113817]: 2024-06-01 02:07:01.03837932 +0000 UTC m=+0.143682444 container create 45f7a302aa67401d63caf7398292b947eb01489eb2e410e36ab7320444e3a3b6 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-activate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-06-01T02:07:01.270 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:01 smithi116 podman[113817]: 2024-06-01 02:07:01.274526817 +0000 UTC m=+0.379829950 container init 45f7a302aa67401d63caf7398292b947eb01489eb2e410e36ab7320444e3a3b6 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-activate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528) 2024-06-01T02:07:01.543 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:01 smithi116 podman[113817]: 2024-06-01 02:07:01.278784658 +0000 UTC m=+0.384087787 container start 45f7a302aa67401d63caf7398292b947eb01489eb2e410e36ab7320444e3a3b6 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-activate, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-3d89acc, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-06-01T02:07:01.543 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:01 smithi116 podman[113817]: 2024-06-01 02:07:01.374405272 +0000 UTC m=+0.479708402 container attach 45f7a302aa67401d63caf7398292b947eb01489eb2e410e36ab7320444e3a3b6 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-activate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240528) 2024-06-01T02:07:02.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:01 smithi116 ceph-mon[100606]: pgmap v89: 161 pgs: 22 stale+active+clean, 139 active+clean; 585 KiB data, 122 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-01T02:07:02.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:01 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:02.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:01 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:07:02.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:01 smithi116 ceph-mon[89502]: pgmap v89: 161 pgs: 22 stale+active+clean, 139 active+clean; 585 KiB data, 122 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-01T02:07:02.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:02.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:07:02.044 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:01 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-activate[113839]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-06-01T02:07:02.044 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:01 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-activate[113839]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-06-01T02:07:02.044 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:01 smithi116 bash[113817]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-06-01T02:07:02.044 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:01 smithi116 bash[113817]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-06-01T02:07:02.044 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:01 smithi116 bash[113817]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-06-01T02:07:02.044 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:01 smithi116 bash[113817]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-06-01T02:07:02.044 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:01 smithi116 bash[113817]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-06-01T02:07:02.044 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:01 smithi116 bash[113817]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-06-01T02:07:02.044 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:01 smithi116 bash[113817]: --> ceph-volume raw activate successful for osd ID: 0 2024-06-01T02:07:02.044 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:01 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-activate[113839]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-06-01T02:07:02.045 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:01 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-activate[113839]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-06-01T02:07:02.045 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:01 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-activate[113839]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-06-01T02:07:02.045 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:01 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-activate[113839]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-06-01T02:07:02.045 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:01 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-activate[113839]: --> ceph-volume raw activate successful for osd ID: 0 2024-06-01T02:07:02.045 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:01 smithi116 podman[113817]: 2024-06-01 02:07:01.758242561 +0000 UTC m=+0.863545697 container died 45f7a302aa67401d63caf7398292b947eb01489eb2e410e36ab7320444e3a3b6 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-activate, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux ) 2024-06-01T02:07:02.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:01 smithi161 ceph-mon[84406]: pgmap v89: 161 pgs: 22 stale+active+clean, 139 active+clean; 585 KiB data, 122 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-01T02:07:02.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:01 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:02.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:01 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:07:02.733 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:02 smithi116 podman[113963]: 2024-06-01 02:07:02.462222494 +0000 UTC m=+0.698237049 container remove 45f7a302aa67401d63caf7398292b947eb01489eb2e410e36ab7320444e3a3b6 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-activate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-01T02:07:02.733 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:02 smithi116 podman[114025]: 2024-06-01 02:07:02.576178333 +0000 UTC m=+0.016840668 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:07:02.733 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:02 smithi116 podman[114025]: 2024-06-01 02:07:02.737959422 +0000 UTC m=+0.178621732 container create b4313d121b9d3a7150b6144d61c0216ab9001ea5798e2726752809ed28f7977d (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-01T02:07:02.985 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:02 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:07:02.985 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:02 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:07:02.985 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:02 smithi116 podman[114025]: 2024-06-01 02:07:02.966006608 +0000 UTC m=+0.406668922 container init b4313d121b9d3a7150b6144d61c0216ab9001ea5798e2726752809ed28f7977d (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-01T02:07:02.985 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:02 smithi116 podman[114025]: 2024-06-01 02:07:02.969303214 +0000 UTC m=+0.409965527 container start b4313d121b9d3a7150b6144d61c0216ab9001ea5798e2726752809ed28f7977d (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-06-01T02:07:03.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:02 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:07:03.293 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:02 smithi116 ceph-osd[114045]: -- 172.21.15.116:0/3750762683 <== mon.1 v2:172.21.15.116:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55754a531680 con 0x55754a4fb000 2024-06-01T02:07:03.293 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:03 smithi116 bash[114025]: b4313d121b9d3a7150b6144d61c0216ab9001ea5798e2726752809ed28f7977d 2024-06-01T02:07:03.293 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:03 smithi116 systemd[1]: Started Ceph osd.0 for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:07:03.793 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0[114041]: 2024-06-01T02:07:03.509+0000 7f9c2088d7c0 -1 Falling back to public interface 2024-06-01T02:07:03.842 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:07:03.505Z caller=manager.go:609 level=warn component="rule manager" group=nodes msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on(instance) group_left(nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi116\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"393f964a-1fb9-11ef-bc9b-c7b262605968\", domainname=\"(none)\", instance=\"smithi116\", job=\"node\", machine=\"x86_64\", nodename=\"smithi116\", release=\"5.14.0-452.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-06-01T02:07:04.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: pgmap v90: 161 pgs: 23 active+undersized+degraded, 40 active+undersized, 98 active+clean; 585 KiB data, 127 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 85/726 objects degraded (11.708%) 2024-06-01T02:07:04.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: Health check failed: Degraded data redundancy: 85/726 objects degraded (11.708%), 23 pgs degraded (PG_DEGRADED) 2024-06-01T02:07:04.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:04.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:04.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:04.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:04.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:04.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:04.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:04.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:04.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:04.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:04.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:04.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:04.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:04.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:04.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:04.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:04.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:04.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:04.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:04.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:04.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:04.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:04.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:04.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:04.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:04.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:04.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:04.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:04.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:04.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:04.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:04.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:04.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:04.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:04.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:03 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:04.209 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: pgmap v90: 161 pgs: 23 active+undersized+degraded, 40 active+undersized, 98 active+clean; 585 KiB data, 127 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 85/726 objects degraded (11.708%) 2024-06-01T02:07:04.209 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: Health check failed: Degraded data redundancy: 85/726 objects degraded (11.708%), 23 pgs degraded (PG_DEGRADED) 2024-06-01T02:07:04.209 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:04.209 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:04.209 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:04.209 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:04.210 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:04.210 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:04.210 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:04.210 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:04.210 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:04.210 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:04.210 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:04.210 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:04.210 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:04.210 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:04.210 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:04.210 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:04.210 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:04.210 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:04.211 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:04.211 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:04.211 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:04.211 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:04.211 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:04.211 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:04.211 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:04.212 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: pgmap v90: 161 pgs: 23 active+undersized+degraded, 40 active+undersized, 98 active+clean; 585 KiB data, 127 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 85/726 objects degraded (11.708%) 2024-06-01T02:07:04.212 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: Health check failed: Degraded data redundancy: 85/726 objects degraded (11.708%), 23 pgs degraded (PG_DEGRADED) 2024-06-01T02:07:04.212 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:04.212 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:04.212 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:04.212 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:04.212 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:04.212 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:04.212 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:04.212 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:04.212 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:04.213 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:04.213 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:04.213 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:04.213 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:04.213 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:04.213 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:04.213 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:04.213 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:04.213 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:04.213 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:04.213 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:04.213 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:04.213 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:04.213 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:04.214 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:04.214 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:04.214 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:04.214 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:04.214 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:04.214 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:04.214 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:04.214 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:04.214 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:04.214 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:04.214 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:04.214 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:04.215 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:04.215 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:04.215 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:04.215 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:04.215 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:04.215 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:04.215 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:04.215 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:04.216 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:04.216 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:03 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:05.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:05.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:05.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:05.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:05.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:05.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:05.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:05.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:05.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:05.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:05.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:05.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:05.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:05.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:05.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:05.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:05.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:05.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:05.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:05.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:05.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:05.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:05.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:05.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:05.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:05.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:05.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:05.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:05.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:05.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:05.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:05.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:05.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:05.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:05.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:05.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:05.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:05.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:05.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:05.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:05.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:05.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:05.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:05.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:05.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:05.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:05.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:05.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:05.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:05.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:05.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:05.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:05.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:05.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:05.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:05.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:05.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:05.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:05.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:05.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:05.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:05.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:05.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:05.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:05.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:05.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:05.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:05.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:05.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:05.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:05.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:05.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:05.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:05.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:05.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:05.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:05.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:05.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:05.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:05.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:05.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:05.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:05.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:05.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:05.054 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:05.054 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:05.054 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:05.054 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:05.054 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:05.054 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:05.054 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:05.054 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:05.054 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:05.054 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:05.054 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:05.054 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:05.054 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:05.055 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:05.055 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:05.055 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:05.055 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:05.055 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:05.055 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:05.055 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:04 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:05.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:05.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:05.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:05.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:05.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:05.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:05.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:05.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:05.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:05.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:05.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:05.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:05.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:05.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:05.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:05.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:05.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:05.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:05.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:05.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:05.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:05.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:05.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:05.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:05.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:05.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:05.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:05.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:05.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:05.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:05.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:05.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:05.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:05.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:05.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:05.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:05.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:05.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:05.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:05.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:05.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:05.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:05.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:05.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:05.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:05.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:05.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:05.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:05.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:05.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:05.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:05.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:04 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:06.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:06.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:06.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:06.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:06.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:06.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:06.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:06.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:06.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: pgmap v91: 161 pgs: 23 active+undersized+degraded, 40 active+undersized, 98 active+clean; 585 KiB data, 127 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 85/726 objects degraded (11.708%) 2024-06-01T02:07:06.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:06.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:06.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:06.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:06.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:06.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:06.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:06.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:06.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:06.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:06.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:06.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:06.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:06.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:06.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:06.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:06.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:06.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:06.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:06.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:06.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:06.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:06.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:06.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:06.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:06.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:06.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:06.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:06.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: pgmap v91: 161 pgs: 23 active+undersized+degraded, 40 active+undersized, 98 active+clean; 585 KiB data, 127 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 85/726 objects degraded (11.708%) 2024-06-01T02:07:06.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:06.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:06.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:06.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:06.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:06.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:06.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:06.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:06.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:06.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:06.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:06.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:06.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:06.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:06.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:06.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:06.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:06.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:06.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:06.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:06.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:06.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:06.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:06.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:06.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:06.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:06.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:06.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:06.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:06.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:06.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:06.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:06.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:06.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:06.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:06.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:06.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:06.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:06.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:06.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:06.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:06.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:06.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:06.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:06.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:06.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:06.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:06.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:06.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:06.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:06.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:06.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:06.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:06.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:06.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:06.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:06.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:06.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:06.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:06.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:06.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:06.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:06.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:06.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:06.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:06.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:06.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:06.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:06.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:06.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:06.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:06.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:06.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:06.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:06.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:06.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:06.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:06.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:06.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:06.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:06.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: pgmap v91: 161 pgs: 23 active+undersized+degraded, 40 active+undersized, 98 active+clean; 585 KiB data, 127 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 85/726 objects degraded (11.708%) 2024-06-01T02:07:06.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:06.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:06.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:06.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:06.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:06.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:06.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:06.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:06.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:06.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:06.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:06.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:06.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:06.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:06.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:06.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:06.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:06.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:06.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:06.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:06.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:06.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:06.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:06.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:06.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:06.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:06.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:06.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:06.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:06.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:06.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:06.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:06.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:06.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:06.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:06.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:06.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:06.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:06.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:06.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:06.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:06.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:06.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:06.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:06.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:06.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:06.366 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0[114041]: 2024-06-01T02:07:06.265+0000 7f9c2088d7c0 -1 osd.0 123 log_to_monitors true 2024-06-01T02:07:07.129 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:07.129 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:07.129 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:07.129 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:07.129 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:07.129 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:07.130 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:07.130 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:07.130 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:07.130 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:07.130 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:07.130 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:07.130 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:07.130 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:07.130 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:07.130 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:07.131 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:07.131 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:07.131 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:07.131 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:07.131 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:07.131 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:07.131 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:07.131 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:07.131 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='osd.0 [v2:172.21.15.116:6800/3747426244,v1:172.21.15.116:6801/3747426244]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-01T02:07:07.132 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:07.132 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:07.132 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:07.132 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:07.132 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:07.132 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:07.132 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:07.133 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:07.133 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:07.133 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:07.133 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:07.133 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:07.133 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:07.133 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:07.133 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:07.133 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:07.133 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:07.133 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:07.133 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:07.134 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:07.134 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:07.134 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:07.134 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:07.134 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:07.134 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:07.134 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:07.134 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:07.134 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:07.134 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='osd.0 [v2:172.21.15.116:6800/3747426244,v1:172.21.15.116:6801/3747426244]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-01T02:07:07.134 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:07.134 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:07.134 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:07.135 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:07.135 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:07.135 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:07.135 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:07.135 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:07.135 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:07.135 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:07.135 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:07.135 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:07.135 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:07.136 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:07.136 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:07.136 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:07.136 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:07.136 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:07.136 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:07.136 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:07.136 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:07.136 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:07.136 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:07.136 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:07.136 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:07.136 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:07.136 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:07.137 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:07.137 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:07.137 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:07.137 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:07.137 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:07.137 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:07.137 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:07.138 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:07.138 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:07.138 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:07.138 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:07.138 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:07.138 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:07.138 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:07.138 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:07.138 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:07.138 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:07.138 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:07.139 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:07.139 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:07.139 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:07.139 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:07.139 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:07.139 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:07.139 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:07.139 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:07.139 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:07.139 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:07.140 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:06 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:07.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:07:06] "GET /metrics HTTP/1.1" 200 38357 "" "Prometheus/2.33.4" 2024-06-01T02:07:07.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:07.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:07.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:07.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:07.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:07.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:07.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:07.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:07.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:07.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:07.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:07.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:07.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:07.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:07.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:07.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:07.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:07.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:07.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:07.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:07.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:07.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:07.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:07.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:07.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='osd.0 [v2:172.21.15.116:6800/3747426244,v1:172.21.15.116:6801/3747426244]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-01T02:07:07.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:07.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:07.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:07.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:07.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:07.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:07.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:07.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:07.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:07.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:07.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:07.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:07.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:07.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:07.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:07.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:07.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:07.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:07.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:07.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:07.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:07.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:07.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:07.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:07.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:07.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:07.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:07.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:07.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:07.200 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:06 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:07.449 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0[114041]: 2024-06-01T02:07:07.225+0000 7f9c11ce4700 -1 osd.0 123 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-01T02:07:08.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:08.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:08.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:08.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:08.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:08.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:08.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:08.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:08.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: pgmap v92: 161 pgs: 23 active+undersized+degraded, 40 active+undersized, 98 active+clean; 585 KiB data, 127 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 85/726 objects degraded (11.708%) 2024-06-01T02:07:08.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:08.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:08.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:08.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:08.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:08.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:08.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:08.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:08.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:08.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:08.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:08.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:08.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:08.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:08.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:08.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:08.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='osd.0 [v2:172.21.15.116:6800/3747426244,v1:172.21.15.116:6801/3747426244]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-01T02:07:08.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: osdmap e126: 8 total, 7 up, 8 in 2024-06-01T02:07:08.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='osd.0 [v2:172.21.15.116:6800/3747426244,v1:172.21.15.116:6801/3747426244]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T02:07:08.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:08.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:08.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:08.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:08.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:08.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:08.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:08.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:08.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:08.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:08.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:08.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:08.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:08.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:08.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:08.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:08.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:08.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:08.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:08.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:08.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:08.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:08.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:08.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:08.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:08.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:08.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:08.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:08.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:08.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:08.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:08.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:08.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:08.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:08.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:08.238 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:08.238 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:08.239 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:08.239 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:08.239 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:08.239 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:08.239 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:08.239 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:08.239 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: pgmap v92: 161 pgs: 23 active+undersized+degraded, 40 active+undersized, 98 active+clean; 585 KiB data, 127 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 85/726 objects degraded (11.708%) 2024-06-01T02:07:08.239 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:08.239 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:08.239 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:08.239 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:08.239 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:08.239 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:08.239 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:08.240 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:08.240 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:08.240 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:08.240 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:08.240 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:08.240 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:08.240 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:08.240 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:08.240 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:08.240 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='osd.0 [v2:172.21.15.116:6800/3747426244,v1:172.21.15.116:6801/3747426244]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-01T02:07:08.240 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: osdmap e126: 8 total, 7 up, 8 in 2024-06-01T02:07:08.240 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='osd.0 [v2:172.21.15.116:6800/3747426244,v1:172.21.15.116:6801/3747426244]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T02:07:08.240 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:08.240 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:08.240 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:08.241 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:08.241 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:08.241 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:08.241 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:08.241 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:08.241 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:08.241 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:08.241 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:08.241 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:08.241 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:08.241 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:08.241 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:08.241 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:08.241 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:08.242 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:08.242 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:08.242 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:08.242 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:08.242 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:08.242 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:08.242 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:08.242 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:08.242 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:08.242 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:08.242 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:08.242 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:08.243 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:08.243 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:08.243 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:08.243 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:08.243 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:08.243 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:08.243 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:08.244 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: pgmap v92: 161 pgs: 23 active+undersized+degraded, 40 active+undersized, 98 active+clean; 585 KiB data, 127 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 85/726 objects degraded (11.708%) 2024-06-01T02:07:08.244 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:08.244 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:08.244 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:08.244 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:08.244 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:08.244 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:08.244 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:08.244 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:08.244 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:08.244 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:08.244 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:08.244 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:08.244 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:08.245 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:08.245 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:08.245 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:08.245 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='osd.0 [v2:172.21.15.116:6800/3747426244,v1:172.21.15.116:6801/3747426244]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-01T02:07:08.245 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: osdmap e126: 8 total, 7 up, 8 in 2024-06-01T02:07:08.245 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='osd.0 [v2:172.21.15.116:6800/3747426244,v1:172.21.15.116:6801/3747426244]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T02:07:08.245 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:08.245 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:08.245 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:08.245 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:08.245 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:08.245 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:08.245 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:08.245 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:08.246 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:08.246 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:08.246 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:08.246 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:08.246 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:08.246 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:08.246 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:08.246 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:08.246 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:08.246 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:08.246 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:08.246 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:08.246 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:08.246 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:08.247 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:08.247 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:08.247 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:08.247 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:08.247 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:08.247 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:08.247 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:08.247 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:08.247 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:08.247 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:08.247 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:08.247 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:08.247 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:08.248 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:08.248 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:08.248 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:08.248 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:08.248 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:08.248 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:08.248 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:09.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:09.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:09.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:09.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:09.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:09.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:09.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:09.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:09.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:09.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:09.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:09.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:09.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:09.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:09.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:09.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:09.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:09.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:09.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:09.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:09.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:09.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:09.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:09.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:09.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:09.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:07:09.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: osd.0 [v2:172.21.15.116:6800/3747426244,v1:172.21.15.116:6801/3747426244] boot 2024-06-01T02:07:09.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: osdmap e127: 8 total, 8 up, 8 in 2024-06-01T02:07:09.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T02:07:09.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:09.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:09.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:09.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:09.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:09.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:09.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:09.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:09.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:09.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:09.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:09.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:09.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:09.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:09.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:09.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:09.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:09.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:09.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:09.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:09.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:09.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:09.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:09.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:09.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:09.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:09.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:09.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:09.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:09.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:08 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:09.255 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:09.255 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:09.255 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:09.255 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:09.255 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:09.255 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:09.255 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:09.255 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:09.255 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:09.255 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:09.255 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:09.255 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:09.255 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:09.256 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:09.256 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:09.256 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:09.256 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:09.256 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:09.256 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:09.256 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:09.256 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:09.256 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:09.256 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:09.256 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:09.256 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:09.256 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:07:09.256 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: osd.0 [v2:172.21.15.116:6800/3747426244,v1:172.21.15.116:6801/3747426244] boot 2024-06-01T02:07:09.256 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: osdmap e127: 8 total, 8 up, 8 in 2024-06-01T02:07:09.257 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T02:07:09.257 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:09.257 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:09.257 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:09.257 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:09.257 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:09.257 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:09.257 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:09.257 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:09.258 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:09.258 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:09.258 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:09.258 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:09.258 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:09.258 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:09.258 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:09.258 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:09.258 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:09.258 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:09.258 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:09.258 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:09.258 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:09.259 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:09.259 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:09.259 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:09.259 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:09.259 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:09.259 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:09.259 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:07:09.259 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: osd.0 [v2:172.21.15.116:6800/3747426244,v1:172.21.15.116:6801/3747426244] boot 2024-06-01T02:07:09.259 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: osdmap e127: 8 total, 8 up, 8 in 2024-06-01T02:07:09.259 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-01T02:07:09.259 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:09.259 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:09.260 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:09.260 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:09.260 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:09.260 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:09.260 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:09.260 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:09.260 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:09.260 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:09.260 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:09.260 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:09.260 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:09.260 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:09.260 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:09.261 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:09.261 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:09.261 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:09.261 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:09.261 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:09.261 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:09.261 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:09.261 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:09.261 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:09.261 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:09.261 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:09.261 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:09.261 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:09.261 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:09.261 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:09.262 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:09.262 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:09.262 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:09.262 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:09.262 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:09.262 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:09.262 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:09.262 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:09.262 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:09.262 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:09.263 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:09.263 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:09.263 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:09.263 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:09.263 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:09.263 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:09.263 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:09.263 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:09.263 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:09.263 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:09.263 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:09.263 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:09.264 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:09.264 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:09.264 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:09.264 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:09.264 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:09.264 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:08 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:10.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:10.007 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:10.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:10.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:10.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:10.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:10.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:10.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:10.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: pgmap v95: 161 pgs: 23 active+undersized+degraded, 40 active+undersized, 98 active+clean; 585 KiB data, 127 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 85/726 objects degraded (11.708%) 2024-06-01T02:07:10.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:10.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:10.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:10.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:10.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.008 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:10.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:10.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:10.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:10.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:10.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:10.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:10.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:10.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: osdmap e128: 8 total, 8 up, 8 in 2024-06-01T02:07:10.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:10.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:10.009 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:10.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:10.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:10.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:10.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:10.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:10.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:10.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:10.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:10.010 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:10.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: pgmap v95: 161 pgs: 23 active+undersized+degraded, 40 active+undersized, 98 active+clean; 585 KiB data, 127 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 85/726 objects degraded (11.708%) 2024-06-01T02:07:10.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:10.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:10.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:10.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:10.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:10.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:10.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:10.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:10.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:10.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:10.011 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:10.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:10.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: osdmap e128: 8 total, 8 up, 8 in 2024-06-01T02:07:10.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:10.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:10.012 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:10.012 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:10.012 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.012 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:10.012 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:10.012 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:10.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:10.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:10.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:10.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:10.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:10.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:10.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:10.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:10.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:10.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:10.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:10.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:10.014 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:10.014 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.014 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.014 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:10.014 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:10.014 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.014 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:10.014 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:10.014 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.014 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.014 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:10.015 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.015 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:10.015 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:10.015 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:10.015 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:10.015 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.015 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:10.015 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:10.015 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:10.015 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:10.015 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.015 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:10.015 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:10.015 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:10.015 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:10.016 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.016 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:10.016 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:10.016 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:10.016 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:10.016 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.016 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.016 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:10.016 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:10.016 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.016 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:10.016 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:10.017 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.017 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:09 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:07:09.887+0000 7fcedaabb700 -1 mgr.server reply reply (16) Device or resource busy unsafe to stop osd(s) at this time (19 PGs are or would become offline) 2024-06-01T02:07:10.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:10.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:10.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:10.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:10.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:10.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:10.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:10.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:10.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:10.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: pgmap v95: 161 pgs: 23 active+undersized+degraded, 40 active+undersized, 98 active+clean; 585 KiB data, 127 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 85/726 objects degraded (11.708%) 2024-06-01T02:07:10.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:10.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:10.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:10.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:10.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:10.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:10.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:10.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:10.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:10.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:10.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:10.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:10.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: osdmap e128: 8 total, 8 up, 8 in 2024-06-01T02:07:10.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:10.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:10.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:10.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:10.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:10.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:10.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:10.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:10.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:10.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:10.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:10.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:10.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:10.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:10.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:10.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:10.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:10.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:10.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:10.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:10.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:10.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:10.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:10.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:10.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:10.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:09 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:11.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:10 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:11.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:10 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:11.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:10 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:11.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:10 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:11.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:10 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:11.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:10 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:11.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:10 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:11.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:10 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:11.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:10 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:11.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:10 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:11.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:10 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:11.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:10 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:11.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:10 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-06-01T02:07:11.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:10 smithi161 ceph-mon[84406]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-06-01T02:07:11.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:10 smithi161 ceph-mon[84406]: Upgrade: unsafe to stop osd(s) at this time (19 PGs are or would become offline) 2024-06-01T02:07:11.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:10 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:11.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:10 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:11.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:10 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:11.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:10 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:11.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:10 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:11.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:10 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:11.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:10 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:11.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:10 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:11.236 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:10 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:11.236 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:10 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:11.236 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:10 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:11.236 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:10 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:11.236 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:10 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-06-01T02:07:11.236 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:10 smithi116 ceph-mon[100606]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-06-01T02:07:11.236 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:10 smithi116 ceph-mon[100606]: Upgrade: unsafe to stop osd(s) at this time (19 PGs are or would become offline) 2024-06-01T02:07:11.236 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:10 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:11.236 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:10 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:11.237 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:10 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:11.237 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:10 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:11.237 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:10 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:11.237 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:10 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:11.237 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:10 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:11.237 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:10 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:11.237 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:10 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:11.237 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:10 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:11.237 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:10 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:11.237 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:10 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:11.237 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:10 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-06-01T02:07:11.237 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:10 smithi116 ceph-mon[89502]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-06-01T02:07:11.238 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:10 smithi116 ceph-mon[89502]: Upgrade: unsafe to stop osd(s) at this time (19 PGs are or would become offline) 2024-06-01T02:07:11.951 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:11 smithi116 ceph-mon[100606]: pgmap v97: 161 pgs: 23 active+undersized+degraded, 40 active+undersized, 98 active+clean; 585 KiB data, 127 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 85/726 objects degraded (11.708%) 2024-06-01T02:07:11.952 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:11 smithi116 ceph-mon[89502]: pgmap v97: 161 pgs: 23 active+undersized+degraded, 40 active+undersized, 98 active+clean; 585 KiB data, 127 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 85/726 objects degraded (11.708%) 2024-06-01T02:07:12.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:11 smithi161 ceph-mon[84406]: pgmap v97: 161 pgs: 23 active+undersized+degraded, 40 active+undersized, 98 active+clean; 585 KiB data, 127 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 85/726 objects degraded (11.708%) 2024-06-01T02:07:13.142 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:12 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:07:13.142 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:12 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:07:13.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:12 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:07:14.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:13 smithi161 ceph-mon[84406]: pgmap v98: 161 pgs: 161 active+clean; 585 KiB data, 128 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:07:14.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:13 smithi161 ceph-mon[84406]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 85/726 objects degraded (11.708%), 23 pgs degraded) 2024-06-01T02:07:14.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:13 smithi161 ceph-mon[84406]: Cluster is now healthy 2024-06-01T02:07:14.223 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:13 smithi116 ceph-mon[100606]: pgmap v98: 161 pgs: 161 active+clean; 585 KiB data, 128 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:07:14.223 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:13 smithi116 ceph-mon[100606]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 85/726 objects degraded (11.708%), 23 pgs degraded) 2024-06-01T02:07:14.223 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:13 smithi116 ceph-mon[100606]: Cluster is now healthy 2024-06-01T02:07:14.224 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:13 smithi116 ceph-mon[89502]: pgmap v98: 161 pgs: 161 active+clean; 585 KiB data, 128 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:07:14.224 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:13 smithi116 ceph-mon[89502]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 85/726 objects degraded (11.708%), 23 pgs degraded) 2024-06-01T02:07:14.224 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:13 smithi116 ceph-mon[89502]: Cluster is now healthy 2024-06-01T02:07:16.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:15 smithi161 ceph-mon[84406]: pgmap v99: 161 pgs: 161 active+clean; 585 KiB data, 128 MiB used, 715 GiB / 715 GiB avail; 673 B/s rd, 0 op/s 2024-06-01T02:07:16.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:16.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:16.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:16.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:16.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:16.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:16.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:16.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:07:16.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:15 smithi116 ceph-mon[89502]: pgmap v99: 161 pgs: 161 active+clean; 585 KiB data, 128 MiB used, 715 GiB / 715 GiB avail; 673 B/s rd, 0 op/s 2024-06-01T02:07:16.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:16.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:16.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:16.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:16.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:16.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:16.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:16.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:07:16.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:15 smithi116 ceph-mon[100606]: pgmap v99: 161 pgs: 161 active+clean; 585 KiB data, 128 MiB used, 715 GiB / 715 GiB avail; 673 B/s rd, 0 op/s 2024-06-01T02:07:16.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:16.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:16.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:16.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:16.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:16.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:16.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:16.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:07:17.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:07:16 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:07:16] "GET /metrics HTTP/1.1" 200 38358 "" "Prometheus/2.33.4" 2024-06-01T02:07:18.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:17 smithi161 ceph-mon[84406]: pgmap v100: 161 pgs: 161 active+clean; 585 KiB data, 128 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:07:18.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:17 smithi116 ceph-mon[100606]: pgmap v100: 161 pgs: 161 active+clean; 585 KiB data, 128 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:07:18.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:17 smithi116 ceph-mon[89502]: pgmap v100: 161 pgs: 161 active+clean; 585 KiB data, 128 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:07:20.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:19 smithi161 ceph-mon[84406]: pgmap v101: 161 pgs: 161 active+clean; 585 KiB data, 128 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:07:20.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:20.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:20.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:20.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:19 smithi116 ceph-mon[100606]: pgmap v101: 161 pgs: 161 active+clean; 585 KiB data, 128 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:07:20.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:20.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:20.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:20.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:19 smithi116 ceph-mon[89502]: pgmap v101: 161 pgs: 161 active+clean; 585 KiB data, 128 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:07:20.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:20.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:20.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:22.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:21 smithi161 ceph-mon[84406]: pgmap v102: 161 pgs: 161 active+clean; 585 KiB data, 128 MiB used, 715 GiB / 715 GiB avail; 882 B/s rd, 0 op/s 2024-06-01T02:07:22.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:22.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:21 smithi116 ceph-mon[89502]: pgmap v102: 161 pgs: 161 active+clean; 585 KiB data, 128 MiB used, 715 GiB / 715 GiB avail; 882 B/s rd, 0 op/s 2024-06-01T02:07:22.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:22.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:21 smithi116 ceph-mon[100606]: pgmap v102: 161 pgs: 161 active+clean; 585 KiB data, 128 MiB used, 715 GiB / 715 GiB avail; 882 B/s rd, 0 op/s 2024-06-01T02:07:22.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:23.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:22 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:07:23.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:22 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:07:23.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:22 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:07:23.932 INFO:teuthology.orchestra.run.smithi116.stdout:true 2024-06-01T02:07:24.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:23 smithi161 ceph-mon[84406]: pgmap v103: 161 pgs: 161 active+clean; 585 KiB data, 128 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:07:24.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:23 smithi116 ceph-mon[89502]: pgmap v103: 161 pgs: 161 active+clean; 585 KiB data, 128 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:07:24.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:23 smithi116 ceph-mon[100606]: pgmap v103: 161 pgs: 161 active+clean; 585 KiB data, 128 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:07:24.579 INFO:teuthology.orchestra.run.smithi116.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-01T02:07:24.579 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi116 smithi116 running 9s ago 16m - - 2024-06-01T02:07:24.579 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi161 smithi161 running 4s ago 15m - - 2024-06-01T02:07:24.579 INFO:teuthology.orchestra.run.smithi116.stdout:alertmanager.a smithi116 *:9093,9094 running (5m) 9s ago 10m 21.2M - 0.23.0 ba2b418f427c e0c297162ded 2024-06-01T02:07:24.580 INFO:teuthology.orchestra.run.smithi116.stdout:grafana.a smithi161 *:3000 running (5m) 4s ago 10m 52.5M - 8.3.5 dad864ee21e9 00b0a78ee88e 2024-06-01T02:07:24.580 INFO:teuthology.orchestra.run.smithi116.stdout:iscsi.foo.smithi116.gjuxzb smithi116 running (4m) 9s ago 9m 51.2M - 3.5 e1d6a67b021e 0e191fe3e3bb 2024-06-01T02:07:24.580 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.x smithi161 *:8443,9283,8765 running (4m) 4s ago 15m 520M - 19.0.0-2946-g3d89accf b045ac545f82 2d7fbb85e5f7 2024-06-01T02:07:24.580 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.y smithi116 *:8443,9283,8765 running (3m) 9s ago 16m 453M - 19.0.0-2946-g3d89accf b045ac545f82 bc9627c27a57 2024-06-01T02:07:24.580 INFO:teuthology.orchestra.run.smithi116.stdout:mon.a smithi116 running (2m) 9s ago 17m 55.7M 2048M 19.0.0-2946-g3d89accf b045ac545f82 b5e9b6f873ff 2024-06-01T02:07:24.580 INFO:teuthology.orchestra.run.smithi116.stdout:mon.b smithi161 running (93s) 4s ago 15m 37.1M 2048M 19.0.0-2946-g3d89accf b045ac545f82 aaa08c03cfc8 2024-06-01T02:07:24.580 INFO:teuthology.orchestra.run.smithi116.stdout:mon.c smithi116 running (109s) 9s ago 15m 42.2M 2048M 19.0.0-2946-g3d89accf b045ac545f82 bd0fb0e603fa 2024-06-01T02:07:24.580 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.a smithi116 *:9100 running (5m) 9s ago 11m 17.2M - 1.3.1 1dbe0e931976 e347748d17bb 2024-06-01T02:07:24.580 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.b smithi161 *:9100 running (5m) 4s ago 11m 17.1M - 1.3.1 1dbe0e931976 6e66182f5a1a 2024-06-01T02:07:24.580 INFO:teuthology.orchestra.run.smithi116.stdout:osd.0 smithi116 running (21s) 9s ago 14m 40.9M 4096M 19.0.0-2946-g3d89accf b045ac545f82 b4313d121b9d 2024-06-01T02:07:24.580 INFO:teuthology.orchestra.run.smithi116.stdout:osd.1 smithi116 running (14m) 9s ago 14m 59.7M 4096M 17.2.0 e1d6a67b021e 6ff7fc5abe58 2024-06-01T02:07:24.580 INFO:teuthology.orchestra.run.smithi116.stdout:osd.2 smithi116 running (14m) 9s ago 14m 55.5M 4096M 17.2.0 e1d6a67b021e 6aecbb10256b 2024-06-01T02:07:24.580 INFO:teuthology.orchestra.run.smithi116.stdout:osd.3 smithi116 running (13m) 9s ago 13m 58.2M 4096M 17.2.0 e1d6a67b021e a802290f5290 2024-06-01T02:07:24.580 INFO:teuthology.orchestra.run.smithi116.stdout:osd.4 smithi161 running (13m) 4s ago 13m 60.0M 4096M 17.2.0 e1d6a67b021e 70b3f89b0bd9 2024-06-01T02:07:24.581 INFO:teuthology.orchestra.run.smithi116.stdout:osd.5 smithi161 running (12m) 4s ago 12m 55.0M 4096M 17.2.0 e1d6a67b021e 68358b78a25d 2024-06-01T02:07:24.581 INFO:teuthology.orchestra.run.smithi116.stdout:osd.6 smithi161 running (12m) 4s ago 12m 54.7M 4096M 17.2.0 e1d6a67b021e 50abeb0551e7 2024-06-01T02:07:24.581 INFO:teuthology.orchestra.run.smithi116.stdout:osd.7 smithi161 running (12m) 4s ago 12m 58.8M 4096M 17.2.0 e1d6a67b021e 6004e6983735 2024-06-01T02:07:24.581 INFO:teuthology.orchestra.run.smithi116.stdout:prometheus.a smithi161 *:9095 running (4m) 4s ago 11m 74.8M - 2.33.4 514e6a882f6e e977f6ca1d73 2024-06-01T02:07:24.581 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi116.vzjrxj smithi116 *:8000 running (9m) 9s ago 9m 76.4M - 17.2.0 e1d6a67b021e df4fe3c1dbc3 2024-06-01T02:07:24.581 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi161.ashqxz smithi161 *:8000 running (9m) 4s ago 9m 77.4M - 17.2.0 e1d6a67b021e 3eecc1f3e2d3 2024-06-01T02:07:24.581 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi116.hokbmf smithi116 *:80 running (9m) 9s ago 9m 76.9M - 17.2.0 e1d6a67b021e 10950ed75dbd 2024-06-01T02:07:24.581 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi161.axjlzw smithi161 *:80 running (9m) 4s ago 9m 76.1M - 17.2.0 e1d6a67b021e 0b5d5ce47772 2024-06-01T02:07:24.977 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:07:24.977 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": { 2024-06-01T02:07:24.977 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 3 2024-06-01T02:07:24.977 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:07:24.977 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": { 2024-06-01T02:07:24.977 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-01T02:07:24.977 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:07:24.977 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": { 2024-06-01T02:07:24.978 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 7, 2024-06-01T02:07:24.978 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 1 2024-06-01T02:07:24.978 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:07:24.978 INFO:teuthology.orchestra.run.smithi116.stdout: "rgw": { 2024-06-01T02:07:24.978 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-06-01T02:07:24.978 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:07:24.978 INFO:teuthology.orchestra.run.smithi116.stdout: "overall": { 2024-06-01T02:07:24.978 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 11, 2024-06-01T02:07:24.978 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 6 2024-06-01T02:07:24.978 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T02:07:24.978 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:07:25.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:24 smithi161 ceph-mon[84406]: from='client.64222 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:07:25.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:24 smithi116 ceph-mon[89502]: from='client.64222 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:07:25.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:24 smithi116 ceph-mon[100606]: from='client.64222 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:07:25.326 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:07:25.326 INFO:teuthology.orchestra.run.smithi116.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-01T02:07:25.327 INFO:teuthology.orchestra.run.smithi116.stdout: "in_progress": true, 2024-06-01T02:07:25.327 INFO:teuthology.orchestra.run.smithi116.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-01T02:07:25.327 INFO:teuthology.orchestra.run.smithi116.stdout: "services_complete": [ 2024-06-01T02:07:25.327 INFO:teuthology.orchestra.run.smithi116.stdout: "mon", 2024-06-01T02:07:25.327 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr" 2024-06-01T02:07:25.327 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-06-01T02:07:25.327 INFO:teuthology.orchestra.run.smithi116.stdout: "progress": "6/23 daemons upgraded", 2024-06-01T02:07:25.327 INFO:teuthology.orchestra.run.smithi116.stdout: "message": "Currently upgrading osd daemons", 2024-06-01T02:07:25.327 INFO:teuthology.orchestra.run.smithi116.stdout: "is_paused": false 2024-06-01T02:07:25.327 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:07:25.742 INFO:teuthology.orchestra.run.smithi116.stdout:HEALTH_OK 2024-06-01T02:07:26.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:25 smithi161 ceph-mon[84406]: from='client.64228 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:07:26.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:25 smithi161 ceph-mon[84406]: from='client.64234 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:07:26.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:25 smithi161 ceph-mon[84406]: pgmap v104: 161 pgs: 161 active+clean; 585 KiB data, 128 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:07:26.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-06-01T02:07:26.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:25 smithi161 ceph-mon[84406]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-06-01T02:07:26.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:25 smithi161 ceph-mon[84406]: Upgrade: osd.1 is safe to restart 2024-06-01T02:07:26.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:25 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/1656126710' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:26.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:25 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/68364252' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:07:26.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:25 smithi116 ceph-mon[89502]: from='client.64228 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:07:26.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:25 smithi116 ceph-mon[89502]: from='client.64234 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:07:26.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:25 smithi116 ceph-mon[89502]: pgmap v104: 161 pgs: 161 active+clean; 585 KiB data, 128 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:07:26.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-06-01T02:07:26.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:25 smithi116 ceph-mon[89502]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-06-01T02:07:26.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:25 smithi116 ceph-mon[89502]: Upgrade: osd.1 is safe to restart 2024-06-01T02:07:26.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:25 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/1656126710' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:26.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:25 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/68364252' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:07:26.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:25 smithi116 ceph-mon[100606]: from='client.64228 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:07:26.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:25 smithi116 ceph-mon[100606]: from='client.64234 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:07:26.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:25 smithi116 ceph-mon[100606]: pgmap v104: 161 pgs: 161 active+clean; 585 KiB data, 128 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:07:26.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-06-01T02:07:26.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:25 smithi116 ceph-mon[100606]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-06-01T02:07:26.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:25 smithi116 ceph-mon[100606]: Upgrade: osd.1 is safe to restart 2024-06-01T02:07:26.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:25 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/1656126710' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:26.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:25 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/68364252' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:07:27.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:07:26 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:07:26] "GET /metrics HTTP/1.1" 200 38358 "" "Prometheus/2.33.4" 2024-06-01T02:07:27.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:26 smithi161 ceph-mon[84406]: from='client.44280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:07:27.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:27.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-01T02:07:27.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:27.209 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:26 smithi116 ceph-mon[89502]: from='client.44280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:07:27.209 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:27.209 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-01T02:07:27.209 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:27.209 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:26 smithi116 ceph-mon[100606]: from='client.44280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:07:27.209 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:27.209 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-01T02:07:27.209 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:28.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:27 smithi161 ceph-mon[84406]: Upgrade: Updating osd.1 2024-06-01T02:07:28.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:27 smithi161 ceph-mon[84406]: Deploying daemon osd.1 on smithi116 2024-06-01T02:07:28.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:27 smithi161 ceph-mon[84406]: pgmap v105: 161 pgs: 161 active+clean; 585 KiB data, 128 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:07:28.268 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:27 smithi116 ceph-mon[100606]: Upgrade: Updating osd.1 2024-06-01T02:07:28.268 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:27 smithi116 ceph-mon[100606]: Deploying daemon osd.1 on smithi116 2024-06-01T02:07:28.268 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:27 smithi116 ceph-mon[100606]: pgmap v105: 161 pgs: 161 active+clean; 585 KiB data, 128 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:07:28.268 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:27 smithi116 ceph-mon[89502]: Upgrade: Updating osd.1 2024-06-01T02:07:28.268 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:27 smithi116 ceph-mon[89502]: Deploying daemon osd.1 on smithi116 2024-06-01T02:07:28.268 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:27 smithi116 ceph-mon[89502]: pgmap v105: 161 pgs: 161 active+clean; 585 KiB data, 128 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:07:30.016 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:29 smithi116 ceph-mon[100606]: pgmap v106: 161 pgs: 161 active+clean; 585 KiB data, 128 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:07:30.016 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:29 smithi116 ceph-mon[89502]: pgmap v106: 161 pgs: 161 active+clean; 585 KiB data, 128 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:07:30.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:29 smithi161 ceph-mon[84406]: pgmap v106: 161 pgs: 161 active+clean; 585 KiB data, 128 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:07:30.271 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:30 smithi116 systemd[1]: Stopping Ceph osd.1 for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:07:30.543 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:30 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1[36127]: 2024-06-01T02:07:30.275+0000 7fde253a6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-01T02:07:30.543 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:30 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1[36127]: 2024-06-01T02:07:30.275+0000 7fde253a6700 -1 osd.1 128 *** Got signal Terminated *** 2024-06-01T02:07:30.543 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:30 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1[36127]: 2024-06-01T02:07:30.275+0000 7fde253a6700 -1 osd.1 128 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-01T02:07:31.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:30 smithi161 ceph-mon[84406]: osd.1 marked itself down and dead 2024-06-01T02:07:31.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:07:31.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:30 smithi116 ceph-mon[89502]: osd.1 marked itself down and dead 2024-06-01T02:07:31.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:07:31.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:30 smithi116 ceph-mon[100606]: osd.1 marked itself down and dead 2024-06-01T02:07:31.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:07:31.682 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:31 smithi116 podman[118097]: 2024-06-01 02:07:31.391973495 +0000 UTC m=+1.223107377 container died 6ff7fc5abe583e0f87f7d0fa560374e0da673ce6cb7018859b793c6b56d2f8cb (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1, distribution-scope=public, build-date=2022-05-03T08:36:31.336870, GIT_BRANCH=HEAD, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-type=git, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, io.k8s.display-name=CentOS Stream 8, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, release=754, com.redhat.component=centos-stream-container, io.buildah.version=1.19.8, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, architecture=x86_64, io.openshift.tags=base centos centos-stream, RELEASE=HEAD, maintainer=Guillaume Abrioux , version=8, vendor=Red Hat, Inc., url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, ceph=True, CEPH_POINT_RELEASE=-17.2.0, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, io.openshift.expose-services=, name=centos-stream) 2024-06-01T02:07:31.965 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:31 smithi116 podman[118097]: 2024-06-01 02:07:31.785247068 +0000 UTC m=+1.616380935 container cleanup 6ff7fc5abe583e0f87f7d0fa560374e0da673ce6cb7018859b793c6b56d2f8cb (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1, vcs-type=git, RELEASE=HEAD, io.k8s.display-name=CentOS Stream 8, maintainer=Guillaume Abrioux , io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.tags=base centos centos-stream, architecture=x86_64, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, GIT_BRANCH=HEAD, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, build-date=2022-05-03T08:36:31.336870, CEPH_POINT_RELEASE=-17.2.0, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, vendor=Red Hat, Inc., vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, distribution-scope=public, io.openshift.expose-services=, name=centos-stream, GIT_CLEAN=True, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, ceph=True, release=754, version=8, com.redhat.component=centos-stream-container, io.buildah.version=1.19.8, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-06-01T02:07:31.965 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:31 smithi116 bash[118097]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1 2024-06-01T02:07:31.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:31 smithi116 ceph-mon[100606]: pgmap v107: 161 pgs: 161 active+clean; 585 KiB data, 128 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:07:31.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:31 smithi116 ceph-mon[100606]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:07:31.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:31 smithi116 ceph-mon[100606]: osdmap e129: 8 total, 7 up, 8 in 2024-06-01T02:07:31.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [0, 4, 7, 3]}]: dispatch 2024-06-01T02:07:31.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 1]}]: dispatch 2024-06-01T02:07:31.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2024-06-01T02:07:31.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2024-06-01T02:07:31.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [0, 4]}]: dispatch 2024-06-01T02:07:31.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 1]}]: dispatch 2024-06-01T02:07:31.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 1]}]: dispatch 2024-06-01T02:07:31.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 5]}]: dispatch 2024-06-01T02:07:31.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 1]}]: dispatch 2024-06-01T02:07:31.967 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:31 smithi116 ceph-mon[89502]: pgmap v107: 161 pgs: 161 active+clean; 585 KiB data, 128 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:07:31.967 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:31 smithi116 ceph-mon[89502]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:07:31.967 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:31 smithi116 ceph-mon[89502]: osdmap e129: 8 total, 7 up, 8 in 2024-06-01T02:07:31.967 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [0, 4, 7, 3]}]: dispatch 2024-06-01T02:07:31.967 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 1]}]: dispatch 2024-06-01T02:07:31.967 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2024-06-01T02:07:31.967 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2024-06-01T02:07:31.967 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [0, 4]}]: dispatch 2024-06-01T02:07:31.967 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 1]}]: dispatch 2024-06-01T02:07:31.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 1]}]: dispatch 2024-06-01T02:07:31.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 5]}]: dispatch 2024-06-01T02:07:31.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 1]}]: dispatch 2024-06-01T02:07:32.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:31 smithi161 ceph-mon[84406]: pgmap v107: 161 pgs: 161 active+clean; 585 KiB data, 128 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:07:32.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:31 smithi161 ceph-mon[84406]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:07:32.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:31 smithi161 ceph-mon[84406]: osdmap e129: 8 total, 7 up, 8 in 2024-06-01T02:07:32.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [0, 4, 7, 3]}]: dispatch 2024-06-01T02:07:32.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 1]}]: dispatch 2024-06-01T02:07:32.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2024-06-01T02:07:32.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]: dispatch 2024-06-01T02:07:32.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [0, 4]}]: dispatch 2024-06-01T02:07:32.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 1]}]: dispatch 2024-06-01T02:07:32.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 1]}]: dispatch 2024-06-01T02:07:32.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 5]}]: dispatch 2024-06-01T02:07:32.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 1]}]: dispatch 2024-06-01T02:07:32.268 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:31 smithi116 podman[118138]: 2024-06-01 02:07:31.970339941 +0000 UTC m=+0.622622263 container remove 6ff7fc5abe583e0f87f7d0fa560374e0da673ce6cb7018859b793c6b56d2f8cb (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, architecture=x86_64, GIT_REPO=https://github.com/ceph/ceph-container.git, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, io.buildah.version=1.19.8, maintainer=Guillaume Abrioux , io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.expose-services=, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, GIT_CLEAN=True, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, name=centos-stream, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, release=754, distribution-scope=public, build-date=2022-05-03T08:36:31.336870, vendor=Red Hat, Inc., vcs-type=git, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, io.openshift.tags=base centos centos-stream, CEPH_POINT_RELEASE=-17.2.0, io.k8s.display-name=CentOS Stream 8, com.redhat.component=centos-stream-container, RELEASE=HEAD, ceph=True, version=8, GIT_BRANCH=HEAD) 2024-06-01T02:07:32.543 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:32 smithi116 podman[118153]: 2024-06-01 02:07:32.373157924 +0000 UTC m=+0.562796597 container died bcb11cc588bd6870dd88a959cf4cc95857e19cdedabab5ded455af6049283ff8 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=quizzical_galois, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, version=8, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, ceph=True, com.redhat.component=centos-stream-container, maintainer=Guillaume Abrioux , name=centos-stream, CEPH_POINT_RELEASE=-17.2.0, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, distribution-scope=public, vcs-type=git, io.openshift.expose-services=, vendor=Red Hat, Inc., GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.19.8, build-date=2022-05-03T08:36:31.336870, io.openshift.tags=base centos centos-stream, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, architecture=x86_64, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, GIT_CLEAN=True, RELEASE=HEAD, release=754, io.k8s.display-name=CentOS Stream 8) 2024-06-01T02:07:33.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [0, 4, 7, 3]}]': finished 2024-06-01T02:07:33.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 1]}]': finished 2024-06-01T02:07:33.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]': finished 2024-06-01T02:07:33.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]': finished 2024-06-01T02:07:33.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [0, 4]}]': finished 2024-06-01T02:07:33.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 1]}]': finished 2024-06-01T02:07:33.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 1]}]': finished 2024-06-01T02:07:33.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 5]}]': finished 2024-06-01T02:07:33.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 1]}]': finished 2024-06-01T02:07:33.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:32 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:07:33.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:32 smithi161 ceph-mon[84406]: osdmap e130: 8 total, 7 up, 8 in 2024-06-01T02:07:33.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [0, 4, 7, 3]}]': finished 2024-06-01T02:07:33.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 1]}]': finished 2024-06-01T02:07:33.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]': finished 2024-06-01T02:07:33.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]': finished 2024-06-01T02:07:33.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [0, 4]}]': finished 2024-06-01T02:07:33.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 1]}]': finished 2024-06-01T02:07:33.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 1]}]': finished 2024-06-01T02:07:33.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 5]}]': finished 2024-06-01T02:07:33.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 1]}]': finished 2024-06-01T02:07:33.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:32 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:07:33.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:32 smithi116 ceph-mon[89502]: osdmap e130: 8 total, 7 up, 8 in 2024-06-01T02:07:33.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [0, 4, 7, 3]}]': finished 2024-06-01T02:07:33.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 1]}]': finished 2024-06-01T02:07:33.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]': finished 2024-06-01T02:07:33.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [7, 2]}]': finished 2024-06-01T02:07:33.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [0, 4]}]': finished 2024-06-01T02:07:33.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 1]}]': finished 2024-06-01T02:07:33.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 1]}]': finished 2024-06-01T02:07:33.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 5]}]': finished 2024-06-01T02:07:33.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 1]}]': finished 2024-06-01T02:07:33.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:32 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:07:33.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:32 smithi116 ceph-mon[100606]: osdmap e130: 8 total, 7 up, 8 in 2024-06-01T02:07:33.567 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:33 smithi116 podman[118334]: 2024-06-01 02:07:33.217777729 +0000 UTC m=+0.026310418 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:07:33.567 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:33 smithi116 podman[118334]: 2024-06-01 02:07:33.335986811 +0000 UTC m=+0.144519541 container create b8d5db60b10d731bc3250fb19a8aebb598f9d950a70cc53e177d3d3a288a86d2 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-06-01T02:07:33.869 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:33 smithi116 podman[118334]: 2024-06-01 02:07:33.571699356 +0000 UTC m=+0.380232040 container init b8d5db60b10d731bc3250fb19a8aebb598f9d950a70cc53e177d3d3a288a86d2 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-01T02:07:33.869 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:33 smithi116 podman[118334]: 2024-06-01 02:07:33.576212885 +0000 UTC m=+0.384745570 container start b8d5db60b10d731bc3250fb19a8aebb598f9d950a70cc53e177d3d3a288a86d2 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_CLEAN=True, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux ) 2024-06-01T02:07:33.869 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:33 smithi116 podman[118334]: 2024-06-01 02:07:33.688867507 +0000 UTC m=+0.497400238 container attach b8d5db60b10d731bc3250fb19a8aebb598f9d950a70cc53e177d3d3a288a86d2 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-06-01T02:07:34.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:33 smithi161 ceph-mon[84406]: pgmap v110: 161 pgs: 2 unknown, 1 active+undersized+degraded, 55 peering, 1 stale+active+clean, 4 active+undersized, 98 active+clean; 585 KiB data, 129 MiB used, 715 GiB / 715 GiB avail; 6/726 objects degraded (0.826%) 2024-06-01T02:07:34.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:33 smithi161 ceph-mon[84406]: Health check failed: Reduced data availability: 9 pgs inactive, 15 pgs peering (PG_AVAILABILITY) 2024-06-01T02:07:34.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:33 smithi161 ceph-mon[84406]: Health check failed: Degraded data redundancy: 6/726 objects degraded (0.826%), 1 pg degraded (PG_DEGRADED) 2024-06-01T02:07:34.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:33 smithi161 ceph-mon[84406]: osdmap e131: 8 total, 7 up, 8 in 2024-06-01T02:07:34.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:33 smithi116 ceph-mon[89502]: pgmap v110: 161 pgs: 2 unknown, 1 active+undersized+degraded, 55 peering, 1 stale+active+clean, 4 active+undersized, 98 active+clean; 585 KiB data, 129 MiB used, 715 GiB / 715 GiB avail; 6/726 objects degraded (0.826%) 2024-06-01T02:07:34.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:33 smithi116 ceph-mon[89502]: Health check failed: Reduced data availability: 9 pgs inactive, 15 pgs peering (PG_AVAILABILITY) 2024-06-01T02:07:34.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:33 smithi116 ceph-mon[89502]: Health check failed: Degraded data redundancy: 6/726 objects degraded (0.826%), 1 pg degraded (PG_DEGRADED) 2024-06-01T02:07:34.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:33 smithi116 ceph-mon[89502]: osdmap e131: 8 total, 7 up, 8 in 2024-06-01T02:07:34.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:33 smithi116 ceph-mon[100606]: pgmap v110: 161 pgs: 2 unknown, 1 active+undersized+degraded, 55 peering, 1 stale+active+clean, 4 active+undersized, 98 active+clean; 585 KiB data, 129 MiB used, 715 GiB / 715 GiB avail; 6/726 objects degraded (0.826%) 2024-06-01T02:07:34.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:33 smithi116 ceph-mon[100606]: Health check failed: Reduced data availability: 9 pgs inactive, 15 pgs peering (PG_AVAILABILITY) 2024-06-01T02:07:34.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:33 smithi116 ceph-mon[100606]: Health check failed: Degraded data redundancy: 6/726 objects degraded (0.826%), 1 pg degraded (PG_DEGRADED) 2024-06-01T02:07:34.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:33 smithi116 ceph-mon[100606]: osdmap e131: 8 total, 7 up, 8 in 2024-06-01T02:07:34.708 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:34 smithi116 podman[118334]: 2024-06-01 02:07:34.403352015 +0000 UTC m=+1.211884700 container died b8d5db60b10d731bc3250fb19a8aebb598f9d950a70cc53e177d3d3a288a86d2 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-01T02:07:35.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:34 smithi116 ceph-mon[89502]: osdmap e132: 8 total, 7 up, 8 in 2024-06-01T02:07:35.042 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:34 smithi116 ceph-mon[100606]: osdmap e132: 8 total, 7 up, 8 in 2024-06-01T02:07:35.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:34 smithi161 ceph-mon[84406]: osdmap e132: 8 total, 7 up, 8 in 2024-06-01T02:07:35.293 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:35 smithi116 podman[118417]: 2024-06-01 02:07:35.046534421 +0000 UTC m=+1.161877161 container remove b8d5db60b10d731bc3250fb19a8aebb598f9d950a70cc53e177d3d3a288a86d2 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-01T02:07:35.293 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:35 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.1.service: Deactivated successfully. 2024-06-01T02:07:35.293 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:35 smithi116 systemd[1]: Stopped Ceph osd.1 for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:07:35.293 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:35 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.1.service: Consumed 9.584s CPU time. 2024-06-01T02:07:35.293 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:35 smithi116 systemd[1]: Starting Ceph osd.1 for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:07:35.667 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:35 smithi116 podman[118626]: 2024-06-01 02:07:35.440927721 +0000 UTC m=+0.021911125 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:07:35.667 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:35 smithi116 podman[118626]: 2024-06-01 02:07:35.547490279 +0000 UTC m=+0.128473676 container create 2cbca2a7f2b85d453ce80d4fb7f692f4af54459f93b71ffe9167f178b536080c (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1-activate, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS) 2024-06-01T02:07:35.934 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:35 smithi116 podman[118626]: 2024-06-01 02:07:35.791788386 +0000 UTC m=+0.372771782 container init 2cbca2a7f2b85d453ce80d4fb7f692f4af54459f93b71ffe9167f178b536080c (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1-activate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-06-01T02:07:35.934 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:35 smithi116 podman[118626]: 2024-06-01 02:07:35.795347982 +0000 UTC m=+0.376331384 container start 2cbca2a7f2b85d453ce80d4fb7f692f4af54459f93b71ffe9167f178b536080c (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240528, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc) 2024-06-01T02:07:35.934 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:35 smithi116 podman[118626]: 2024-06-01 02:07:35.866651608 +0000 UTC m=+0.447635020 container attach 2cbca2a7f2b85d453ce80d4fb7f692f4af54459f93b71ffe9167f178b536080c (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1-activate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True) 2024-06-01T02:07:36.187 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:35 smithi116 ceph-mon[100606]: pgmap v113: 161 pgs: 2 unknown, 1 active+undersized+degraded, 55 peering, 1 stale+active+clean, 4 active+undersized, 98 active+clean; 585 KiB data, 129 MiB used, 715 GiB / 715 GiB avail; 6/726 objects degraded (0.826%) 2024-06-01T02:07:36.187 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:36.187 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:36.187 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:36.187 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:35 smithi116 ceph-mon[89502]: pgmap v113: 161 pgs: 2 unknown, 1 active+undersized+degraded, 55 peering, 1 stale+active+clean, 4 active+undersized, 98 active+clean; 585 KiB data, 129 MiB used, 715 GiB / 715 GiB avail; 6/726 objects degraded (0.826%) 2024-06-01T02:07:36.187 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:36.187 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:36.188 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:36.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:35 smithi161 ceph-mon[84406]: pgmap v113: 161 pgs: 2 unknown, 1 active+undersized+degraded, 55 peering, 1 stale+active+clean, 4 active+undersized, 98 active+clean; 585 KiB data, 129 MiB used, 715 GiB / 715 GiB avail; 6/726 objects degraded (0.826%) 2024-06-01T02:07:36.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:36.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:36.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:36.543 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:36 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1-activate[118687]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-06-01T02:07:36.543 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:36 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1-activate[118687]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-06-01T02:07:36.543 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:36 smithi116 bash[118626]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-06-01T02:07:36.543 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:36 smithi116 bash[118626]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-06-01T02:07:36.544 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:36 smithi116 bash[118626]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-06-01T02:07:36.544 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:36 smithi116 bash[118626]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-06-01T02:07:36.544 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:36 smithi116 bash[118626]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-06-01T02:07:36.544 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:36 smithi116 bash[118626]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-06-01T02:07:36.544 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:36 smithi116 bash[118626]: --> ceph-volume raw activate successful for osd ID: 1 2024-06-01T02:07:36.544 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:36 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1-activate[118687]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-06-01T02:07:36.544 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:36 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1-activate[118687]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-06-01T02:07:36.544 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:36 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1-activate[118687]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-06-01T02:07:36.544 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:36 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1-activate[118687]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-06-01T02:07:36.544 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:36 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1-activate[118687]: --> ceph-volume raw activate successful for osd ID: 1 2024-06-01T02:07:36.544 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:36 smithi116 podman[118626]: 2024-06-01 02:07:36.340555851 +0000 UTC m=+0.921539250 container died 2cbca2a7f2b85d453ce80d4fb7f692f4af54459f93b71ffe9167f178b536080c (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1-activate, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-06-01T02:07:37.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:36 smithi161 ceph-mon[84406]: pgmap v114: 161 pgs: 6 active+clean+remapped, 2 unknown, 19 active+undersized+degraded, 38 active+undersized, 96 active+clean; 585 KiB data, 129 MiB used, 715 GiB / 715 GiB avail; 93/726 objects degraded (12.810%); 15/726 objects misplaced (2.066%); 59 B/s, 1 objects/s recovering 2024-06-01T02:07:37.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:07:36 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:07:36] "GET /metrics HTTP/1.1" 200 38354 "" "Prometheus/2.33.4" 2024-06-01T02:07:37.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:36 smithi116 ceph-mon[89502]: pgmap v114: 161 pgs: 6 active+clean+remapped, 2 unknown, 19 active+undersized+degraded, 38 active+undersized, 96 active+clean; 585 KiB data, 129 MiB used, 715 GiB / 715 GiB avail; 93/726 objects degraded (12.810%); 15/726 objects misplaced (2.066%); 59 B/s, 1 objects/s recovering 2024-06-01T02:07:37.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:36 smithi116 ceph-mon[100606]: pgmap v114: 161 pgs: 6 active+clean+remapped, 2 unknown, 19 active+undersized+degraded, 38 active+undersized, 96 active+clean; 585 KiB data, 129 MiB used, 715 GiB / 715 GiB avail; 93/726 objects degraded (12.810%); 15/726 objects misplaced (2.066%); 59 B/s, 1 objects/s recovering 2024-06-01T02:07:37.793 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:37 smithi116 podman[118869]: 2024-06-01 02:07:37.475332142 +0000 UTC m=+1.182485711 container remove 2cbca2a7f2b85d453ce80d4fb7f692f4af54459f93b71ffe9167f178b536080c (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1-activate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-06-01T02:07:37.793 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:37 smithi116 podman[118983]: 2024-06-01 02:07:37.592166897 +0000 UTC m=+0.018816534 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:07:37.793 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:37 smithi116 podman[118983]: 2024-06-01 02:07:37.70958549 +0000 UTC m=+0.136235119 container create dccdbeac5aaed8e1335f250712c904f1e1cb75ba98c37420ef233be25adcfd36 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-01T02:07:38.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:37 smithi161 ceph-mon[84406]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs inactive, 15 pgs peering) 2024-06-01T02:07:38.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:37 smithi116 ceph-mon[89502]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs inactive, 15 pgs peering) 2024-06-01T02:07:38.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:37 smithi116 ceph-mon[100606]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs inactive, 15 pgs peering) 2024-06-01T02:07:38.293 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:37 smithi116 podman[118983]: 2024-06-01 02:07:37.945293605 +0000 UTC m=+0.371943234 container init dccdbeac5aaed8e1335f250712c904f1e1cb75ba98c37420ef233be25adcfd36 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , ceph=True) 2024-06-01T02:07:38.293 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:37 smithi116 podman[118983]: 2024-06-01 02:07:37.948796038 +0000 UTC m=+0.375445670 container start dccdbeac5aaed8e1335f250712c904f1e1cb75ba98c37420ef233be25adcfd36 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-01T02:07:38.293 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:37 smithi116 ceph-osd[119026]: -- 172.21.15.116:0/3299245922 <== mon.1 v2:172.21.15.116:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560f5c1af680 con 0x560f5c179000 2024-06-01T02:07:38.294 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:38 smithi116 bash[118983]: dccdbeac5aaed8e1335f250712c904f1e1cb75ba98c37420ef233be25adcfd36 2024-06-01T02:07:38.294 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:38 smithi116 systemd[1]: Started Ceph osd.1 for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:07:38.694 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:38 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1[119010]: 2024-06-01T02:07:38.493+0000 7f824d72c7c0 -1 Falling back to public interface 2024-06-01T02:07:39.274 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:39.274 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:39.275 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:39.275 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:39.275 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:39.275 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:39.275 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:39.275 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:39.275 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:39.275 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:39.275 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:39.275 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:39.275 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:39.275 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:39.275 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:39.276 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:39.276 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:39.276 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:39.276 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:39.276 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:39.276 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:39.276 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:39.276 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:39.276 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:39.276 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:39.276 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:39.276 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:39.276 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:39.277 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:39.277 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:39.277 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:39.277 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:39.277 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:39.277 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:39.277 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:39.277 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:39.277 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:39.277 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:39.278 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:39.278 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:39.278 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:39.278 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:39.278 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:39.278 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:39.278 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:39.278 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:39.278 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:39.278 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:39.278 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:39.278 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:39.278 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:39.279 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:39.279 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:39.279 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:39.279 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:39.279 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:39.279 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:39.279 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:39.279 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:39.279 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: pgmap v115: 161 pgs: 8 active+clean+remapped, 19 active+undersized+degraded, 38 active+undersized, 96 active+clean; 585 KiB data, 129 MiB used, 715 GiB / 715 GiB avail; 93/726 objects degraded (12.810%); 24/726 objects misplaced (3.306%); 51 B/s, 1 objects/s recovering 2024-06-01T02:07:39.279 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:39.280 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:39.280 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:39.280 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:39.280 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:39.281 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:39.281 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:39.281 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:39.281 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:39.281 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:39.281 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:39.281 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:39.281 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:39.281 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:39.281 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:39.282 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:39.282 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: pgmap v115: 161 pgs: 8 active+clean+remapped, 19 active+undersized+degraded, 38 active+undersized, 96 active+clean; 585 KiB data, 129 MiB used, 715 GiB / 715 GiB avail; 93/726 objects degraded (12.810%); 24/726 objects misplaced (3.306%); 51 B/s, 1 objects/s recovering 2024-06-01T02:07:39.285 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:39.285 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:39.285 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:39.286 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:39.286 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:39.286 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:39.286 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:39.286 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:39.286 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:39.286 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:39.286 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:39.286 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:39.286 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:39.286 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:39.286 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:39.434 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:39.434 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:39.435 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:39.435 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:39.435 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:39.435 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:39.435 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:39.435 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:39.435 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:39.435 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:39.435 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:39.435 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:39.435 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:39.435 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:39.435 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:39.436 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:39.436 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:39.436 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:39.436 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:39.436 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:39.436 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:39.436 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:39.436 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:39.436 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:39.436 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:39.436 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:39.436 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:39.436 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:39.436 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:39.437 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:39.437 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: pgmap v115: 161 pgs: 8 active+clean+remapped, 19 active+undersized+degraded, 38 active+undersized, 96 active+clean; 585 KiB data, 129 MiB used, 715 GiB / 715 GiB avail; 93/726 objects degraded (12.810%); 24/726 objects misplaced (3.306%); 51 B/s, 1 objects/s recovering 2024-06-01T02:07:39.437 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:39.437 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:39.437 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:39.437 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:39.437 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:39.437 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:39.437 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:39.437 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:39.437 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:39.437 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:39.437 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:39.437 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:39.437 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:39.438 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:39.438 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:40.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:40.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:40.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:40.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:40.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:40.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:40.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:40.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:40.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:40.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:40.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:40.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:40.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:40.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:40.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:40.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:40.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:40.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:40.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:40.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:40.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:40.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:40.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:40.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:40.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:40.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:40.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:40.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:40.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:40.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:40.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:40.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:40.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:40.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:40.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:40.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:40.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:40.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:40.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:40.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:40.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:40.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:40.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:40.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:40.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:40.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:40.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:40.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:40.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:40.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:40.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:40.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:40.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:40.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:40.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:40.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:40.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:40.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:40.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:40.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:40.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:40.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:40.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:40.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:40.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:40.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:40.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:40.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:40.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:40.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:40.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:40.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:40.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:40.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:40.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:40.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:40.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:40.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:40.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:40.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:40.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:40.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:40.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:40.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:40.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:40.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:40.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:40.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:40.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:40.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:40.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:40.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:40.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:40.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:40.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:40.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:40.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:40.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:40.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:40.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:40.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:40.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:40.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:40.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:40.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:40.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:40.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:40.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:40.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:40.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:40.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:40.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:40.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:40.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:40.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:40.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:40.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:40.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:40.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:40.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:40.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:40.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:40.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:40.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:40.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:40.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:40.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:40.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:40.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:40.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:40.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:40.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:40.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:40.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:40.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:40.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:40.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:40.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:40.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:40.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:40.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.149 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:41.149 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.150 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.150 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.150 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.150 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.150 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:41.150 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.150 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.150 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.150 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.150 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.150 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:41.150 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.151 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.151 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.151 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.151 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.151 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:41.151 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.151 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.151 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.151 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.151 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.151 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:41.151 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.151 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.151 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.152 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:41.152 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.152 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.152 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.152 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.152 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.152 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:41.152 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.152 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.153 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.153 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.153 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.153 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:41.153 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.153 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.153 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.153 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.153 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.153 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:41.153 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.153 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.153 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.153 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.154 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.154 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:41.154 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.154 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.154 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.154 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.155 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.155 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:41.155 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.155 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.155 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.155 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.155 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.155 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:41.155 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.155 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.155 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.155 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.155 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.156 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:41.156 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.156 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.156 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.156 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.156 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.156 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:41.156 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: pgmap v116: 161 pgs: 8 active+clean+remapped, 19 active+undersized+degraded, 38 active+undersized, 96 active+clean; 585 KiB data, 129 MiB used, 715 GiB / 715 GiB avail; 93/726 objects degraded (12.810%); 24/726 objects misplaced (3.306%); 44 B/s, 1 objects/s recovering 2024-06-01T02:07:41.156 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.156 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.156 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.156 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.156 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.156 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:41.157 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.157 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.157 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.157 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.157 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.157 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.157 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.157 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.157 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.157 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.158 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.158 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.158 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:41.158 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.158 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.158 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.158 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.158 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.158 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:41.158 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.158 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.158 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.159 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.159 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.159 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:41.159 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.159 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.159 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.159 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.159 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.159 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:41.159 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: pgmap v116: 161 pgs: 8 active+clean+remapped, 19 active+undersized+degraded, 38 active+undersized, 96 active+clean; 585 KiB data, 129 MiB used, 715 GiB / 715 GiB avail; 93/726 objects degraded (12.810%); 24/726 objects misplaced (3.306%); 44 B/s, 1 objects/s recovering 2024-06-01T02:07:41.159 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.159 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.159 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.159 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.159 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.160 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:41.160 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.160 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.160 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.160 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.160 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.160 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.160 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.160 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.160 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.160 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:41.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:41.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:41.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:41.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:41.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:41.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:41.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:41.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:41.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: pgmap v116: 161 pgs: 8 active+clean+remapped, 19 active+undersized+degraded, 38 active+undersized, 96 active+clean; 585 KiB data, 129 MiB used, 715 GiB / 715 GiB avail; 93/726 objects degraded (12.810%); 24/726 objects misplaced (3.306%); 44 B/s, 1 objects/s recovering 2024-06-01T02:07:41.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:41.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:41.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:41.450 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:41.450 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:41.450 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:41.455 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1[119010]: 2024-06-01T02:07:41.255+0000 7f824d72c7c0 -1 osd.1 0 read_superblock omap replica is missing. 2024-06-01T02:07:41.456 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:41 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1[119010]: 2024-06-01T02:07:41.295+0000 7f824d72c7c0 -1 osd.1 128 log_to_monitors true 2024-06-01T02:07:42.261 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:42.261 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:42.261 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:42.262 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:42.262 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:42.262 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:42.262 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:42.262 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:42.262 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='osd.1 [v2:172.21.15.116:6808/3396891614,v1:172.21.15.116:6809/3396891614]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-01T02:07:42.262 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: Health check update: Degraded data redundancy: 93/726 objects degraded (12.810%), 19 pgs degraded (PG_DEGRADED) 2024-06-01T02:07:42.262 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:42.262 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:42.262 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:42.262 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:42.262 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:42.263 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:42.263 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:42.263 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:42.263 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:42.263 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:42.263 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:42.263 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:42.263 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:42.263 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:42.263 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:42.263 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:42.263 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:42.263 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:42.264 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:42.264 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:42.264 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:42.264 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:42.264 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:42.264 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:42.264 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:42.264 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:42.265 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='osd.1 [v2:172.21.15.116:6808/3396891614,v1:172.21.15.116:6809/3396891614]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-01T02:07:42.265 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: Health check update: Degraded data redundancy: 93/726 objects degraded (12.810%), 19 pgs degraded (PG_DEGRADED) 2024-06-01T02:07:42.265 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:42.265 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:42.265 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:42.265 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:42.265 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:42.265 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:42.265 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:42.265 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:42.265 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:42.265 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:42.265 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:42.266 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:42.266 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:42.266 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:42.266 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:42.266 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:42.266 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:42.266 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:42.266 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:42.266 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:42.266 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:42.266 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:42.266 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:42.266 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:42.266 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:42.267 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:42.267 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:42.267 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:42.267 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:42.267 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:42.267 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:42.267 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:42.267 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:42.267 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:42.267 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:42.267 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:42.267 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:42.267 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:42.267 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:42.268 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:42.268 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:42.268 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:42.268 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:07:42.268 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:42.268 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:42.268 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:42.268 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:42.268 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:42.268 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:42.268 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:42.268 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:42.269 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:42.269 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:42.269 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:42.269 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:42.269 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:42.269 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:42.269 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:42.269 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:42.269 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:42.269 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:42.269 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:42.269 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:42.269 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:42.269 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:42.270 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:42.270 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:42.270 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:42.270 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:42.270 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:42.270 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:42.270 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:42.270 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:07:42.270 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:42.270 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:42.270 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:42.270 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:42.270 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:42.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:42.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:42.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:42.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:42.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:42.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:42.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:42.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:42.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='osd.1 [v2:172.21.15.116:6808/3396891614,v1:172.21.15.116:6809/3396891614]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-01T02:07:42.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: Health check update: Degraded data redundancy: 93/726 objects degraded (12.810%), 19 pgs degraded (PG_DEGRADED) 2024-06-01T02:07:42.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:42.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:42.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:42.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:42.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:42.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:42.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:42.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:42.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:42.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:42.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:42.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:42.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:42.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:42.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:42.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:42.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:42.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:42.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:42.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:42.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:42.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:42.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:42.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:42.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:42.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:42.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:42.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:42.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:42.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:42.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:42.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:42.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:42.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:42.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:42.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:42.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:42.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:42.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:42.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:42.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:42.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:42.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:07:42.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:42.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:42.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:42.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:42.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:42.543 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:07:42 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1[119010]: 2024-06-01T02:07:42.264+0000 7f823eb83700 -1 osd.1 128 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-01T02:07:43.370 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:43.370 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:43.371 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:43.371 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:43.371 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:43.371 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:43.371 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:43.371 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:43.371 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:43.371 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:43.371 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:43.371 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:43.371 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:43.371 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='osd.1 [v2:172.21.15.116:6808/3396891614,v1:172.21.15.116:6809/3396891614]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-01T02:07:43.371 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: osdmap e133: 8 total, 7 up, 8 in 2024-06-01T02:07:43.372 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='osd.1 [v2:172.21.15.116:6808/3396891614,v1:172.21.15.116:6809/3396891614]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T02:07:43.372 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:43.372 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:43.372 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:43.372 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:43.372 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:43.372 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:43.372 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:43.372 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:43.372 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:43.372 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:43.372 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:43.372 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:43.373 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:43.373 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:43.373 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:43.373 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:43.373 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:43.373 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:43.373 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:43.373 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:43.373 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:43.373 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:43.373 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:43.374 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:43.374 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:43.374 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='osd.1 [v2:172.21.15.116:6808/3396891614,v1:172.21.15.116:6809/3396891614]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-01T02:07:43.374 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: osdmap e133: 8 total, 7 up, 8 in 2024-06-01T02:07:43.374 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='osd.1 [v2:172.21.15.116:6808/3396891614,v1:172.21.15.116:6809/3396891614]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T02:07:43.374 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:43.374 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:43.374 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:43.374 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:43.374 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:43.375 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:43.375 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:43.375 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:43.375 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:43.375 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:43.375 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:43.375 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:43.375 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:43.376 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:43.376 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:43.376 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:43.376 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:43.376 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:43.376 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:43.376 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:43.376 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:43.376 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:43.376 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:43.376 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:43.376 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:43.376 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:43.377 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:43.377 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:43.377 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:43.377 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:43.377 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: pgmap v118: 161 pgs: 8 active+clean+remapped, 19 active+undersized+degraded, 38 active+undersized, 96 active+clean; 585 KiB data, 140 MiB used, 715 GiB / 715 GiB avail; 115 B/s rd, 0 op/s; 93/726 objects degraded (12.810%); 24/726 objects misplaced (3.306%); 39 B/s, 1 objects/s recovering 2024-06-01T02:07:43.377 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:43.377 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:43.377 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:43.377 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:43.377 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:43.378 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:43.378 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:43.378 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:43.378 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:43.378 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:43.378 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:43.378 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:43.378 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:43.378 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:43.379 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:43.379 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:43.379 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:43.379 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:43.379 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:43.379 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:43.379 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:43.379 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:43.379 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:43.380 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:43.380 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:43.380 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:43.380 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:43.380 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:43.380 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:43.380 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:43.380 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:43.380 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:43.380 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:43.380 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:43.380 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: pgmap v118: 161 pgs: 8 active+clean+remapped, 19 active+undersized+degraded, 38 active+undersized, 96 active+clean; 585 KiB data, 140 MiB used, 715 GiB / 715 GiB avail; 115 B/s rd, 0 op/s; 93/726 objects degraded (12.810%); 24/726 objects misplaced (3.306%); 39 B/s, 1 objects/s recovering 2024-06-01T02:07:43.380 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:43.380 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:43.380 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:43.381 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:43.381 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:43.381 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:43.381 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:43.381 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:43.381 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:43.381 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:43.381 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:43.381 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:43.381 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:43.381 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:43.381 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:43.381 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:43.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:43.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:43.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:43.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:43.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:43.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:43.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:43.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:43.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:43.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:43.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:43.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:43.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:43.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='osd.1 [v2:172.21.15.116:6808/3396891614,v1:172.21.15.116:6809/3396891614]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-01T02:07:43.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: osdmap e133: 8 total, 7 up, 8 in 2024-06-01T02:07:43.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='osd.1 [v2:172.21.15.116:6808/3396891614,v1:172.21.15.116:6809/3396891614]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T02:07:43.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:43.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:43.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:43.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:43.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:43.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:43.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:43.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:43.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:43.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:43.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:43.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:43.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:43.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:43.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:43.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:43.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:43.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:43.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:43.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:43.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:43.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:43.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:43.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:43.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:43.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:43.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:43.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:43.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:43.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:43.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: pgmap v118: 161 pgs: 8 active+clean+remapped, 19 active+undersized+degraded, 38 active+undersized, 96 active+clean; 585 KiB data, 140 MiB used, 715 GiB / 715 GiB avail; 115 B/s rd, 0 op/s; 93/726 objects degraded (12.810%); 24/726 objects misplaced (3.306%); 39 B/s, 1 objects/s recovering 2024-06-01T02:07:43.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:43.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:43.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:43.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:43.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:43.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:43.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:43.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:43.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:43.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:43.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:43.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:43.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:43.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:43.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:43.450 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:44.109 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:07:43 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:07:43.754+0000 7fcedaabb700 -1 mgr.server reply reply (16) Device or resource busy unsafe to stop osd(s) at this time (10 PGs are or would become offline) 2024-06-01T02:07:44.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:44.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:44.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:44.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:44.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:44.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:44.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:44.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:44.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:07:44.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: osd.1 [v2:172.21.15.116:6808/3396891614,v1:172.21.15.116:6809/3396891614] boot 2024-06-01T02:07:44.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: osdmap e134: 8 total, 8 up, 8 in 2024-06-01T02:07:44.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T02:07:44.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:44.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:44.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:44.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:44.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:44.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:44.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:44.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:44.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:44.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:44.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:44.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:44.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:44.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:44.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:44.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:44.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:44.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:44.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:44.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:44.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:44.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:44.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:44.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:44.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:44.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:44.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: pgmap v120: 161 pgs: 8 active+clean+remapped, 19 active+undersized+degraded, 38 active+undersized, 96 active+clean; 585 KiB data, 140 MiB used, 715 GiB / 715 GiB avail; 747 B/s rd, 0 op/s; 93/726 objects degraded (12.810%); 24/726 objects misplaced (3.306%) 2024-06-01T02:07:44.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:44.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:44.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:44.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:44.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:44.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-06-01T02:07:44.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-06-01T02:07:44.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: Upgrade: unsafe to stop osd(s) at this time (10 PGs are or would become offline) 2024-06-01T02:07:44.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:44 smithi161 ceph-mon[84406]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-06-01T02:07:44.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:44.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:44.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:44.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:44.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:44.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:44.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:44.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:44.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:07:44.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: osd.1 [v2:172.21.15.116:6808/3396891614,v1:172.21.15.116:6809/3396891614] boot 2024-06-01T02:07:44.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: osdmap e134: 8 total, 8 up, 8 in 2024-06-01T02:07:44.450 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T02:07:44.450 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:44.450 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:44.450 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:44.450 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:44.450 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:44.450 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:44.450 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:44.450 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:44.450 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:44.450 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:44.450 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:44.451 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:44.451 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:44.451 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:44.451 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:44.451 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:44.451 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:44.451 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:44.452 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:44.452 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:44.452 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:44.452 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:44.452 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:44.452 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:44.452 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:44.452 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:44.453 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:07:44.453 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: osd.1 [v2:172.21.15.116:6808/3396891614,v1:172.21.15.116:6809/3396891614] boot 2024-06-01T02:07:44.453 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: osdmap e134: 8 total, 8 up, 8 in 2024-06-01T02:07:44.453 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-01T02:07:44.453 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:44.453 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:44.453 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:44.453 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:44.453 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:44.453 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:44.453 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:44.453 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:44.453 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:44.454 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:44.454 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:44.454 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:44.454 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:44.454 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:44.454 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:44.454 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:44.454 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:44.454 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:44.454 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:44.454 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:44.455 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:44.455 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:44.455 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:44.455 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:44.455 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:44.455 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:44.455 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: pgmap v120: 161 pgs: 8 active+clean+remapped, 19 active+undersized+degraded, 38 active+undersized, 96 active+clean; 585 KiB data, 140 MiB used, 715 GiB / 715 GiB avail; 747 B/s rd, 0 op/s; 93/726 objects degraded (12.810%); 24/726 objects misplaced (3.306%) 2024-06-01T02:07:44.455 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:44.455 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:44.455 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:44.455 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:44.455 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:44.456 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-06-01T02:07:44.456 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-06-01T02:07:44.456 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: Upgrade: unsafe to stop osd(s) at this time (10 PGs are or would become offline) 2024-06-01T02:07:44.456 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[100606]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-06-01T02:07:44.456 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:44.456 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:44.456 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:07:44.456 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:07:44.456 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:44.456 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:07:44.456 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:44.457 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:44.457 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: pgmap v120: 161 pgs: 8 active+clean+remapped, 19 active+undersized+degraded, 38 active+undersized, 96 active+clean; 585 KiB data, 140 MiB used, 715 GiB / 715 GiB avail; 747 B/s rd, 0 op/s; 93/726 objects degraded (12.810%); 24/726 objects misplaced (3.306%) 2024-06-01T02:07:44.457 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:44.457 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:07:44.457 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:44.457 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:44.457 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:44.457 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-06-01T02:07:44.457 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-06-01T02:07:44.457 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: Upgrade: unsafe to stop osd(s) at this time (10 PGs are or would become offline) 2024-06-01T02:07:44.457 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:44 smithi116 ceph-mon[89502]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-06-01T02:07:45.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:45 smithi116 ceph-mon[89502]: osdmap e135: 8 total, 8 up, 8 in 2024-06-01T02:07:45.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:45 smithi116 ceph-mon[100606]: osdmap e135: 8 total, 8 up, 8 in 2024-06-01T02:07:45.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:45 smithi161 ceph-mon[84406]: osdmap e135: 8 total, 8 up, 8 in 2024-06-01T02:07:46.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:46 smithi116 ceph-mon[89502]: osdmap e136: 8 total, 8 up, 8 in 2024-06-01T02:07:46.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:46 smithi116 ceph-mon[89502]: pgmap v123: 161 pgs: 8 active+clean+remapped, 19 active+undersized+degraded, 38 active+undersized, 96 active+clean; 585 KiB data, 140 MiB used, 715 GiB / 715 GiB avail; 93/726 objects degraded (12.810%); 24/726 objects misplaced (3.306%) 2024-06-01T02:07:46.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:46.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:07:46.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:46.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:46 smithi116 ceph-mon[100606]: osdmap e136: 8 total, 8 up, 8 in 2024-06-01T02:07:46.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:46 smithi116 ceph-mon[100606]: pgmap v123: 161 pgs: 8 active+clean+remapped, 19 active+undersized+degraded, 38 active+undersized, 96 active+clean; 585 KiB data, 140 MiB used, 715 GiB / 715 GiB avail; 93/726 objects degraded (12.810%); 24/726 objects misplaced (3.306%) 2024-06-01T02:07:46.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:46.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:07:46.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:46.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:46 smithi161 ceph-mon[84406]: osdmap e136: 8 total, 8 up, 8 in 2024-06-01T02:07:46.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:46 smithi161 ceph-mon[84406]: pgmap v123: 161 pgs: 8 active+clean+remapped, 19 active+undersized+degraded, 38 active+undersized, 96 active+clean; 585 KiB data, 140 MiB used, 715 GiB / 715 GiB avail; 93/726 objects degraded (12.810%); 24/726 objects misplaced (3.306%) 2024-06-01T02:07:46.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:46.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:07:46.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:47.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:07:46 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:07:46] "GET /metrics HTTP/1.1" 200 38295 "" "Prometheus/2.33.4" 2024-06-01T02:07:47.495 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:47 smithi116 ceph-mon[100606]: osdmap e137: 8 total, 8 up, 8 in 2024-06-01T02:07:47.496 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:47 smithi116 ceph-mon[89502]: osdmap e137: 8 total, 8 up, 8 in 2024-06-01T02:07:47.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:47 smithi161 ceph-mon[84406]: osdmap e137: 8 total, 8 up, 8 in 2024-06-01T02:07:48.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:48 smithi116 ceph-mon[100606]: osdmap e138: 8 total, 8 up, 8 in 2024-06-01T02:07:48.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:48 smithi116 ceph-mon[100606]: pgmap v126: 161 pgs: 1 peering, 160 active+clean; 585 KiB data, 141 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 332 B/s, 4 objects/s recovering 2024-06-01T02:07:48.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:48 smithi116 ceph-mon[89502]: osdmap e138: 8 total, 8 up, 8 in 2024-06-01T02:07:48.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:48 smithi116 ceph-mon[89502]: pgmap v126: 161 pgs: 1 peering, 160 active+clean; 585 KiB data, 141 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 332 B/s, 4 objects/s recovering 2024-06-01T02:07:48.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:48 smithi161 ceph-mon[84406]: osdmap e138: 8 total, 8 up, 8 in 2024-06-01T02:07:48.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:48 smithi161 ceph-mon[84406]: pgmap v126: 161 pgs: 1 peering, 160 active+clean; 585 KiB data, 141 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 332 B/s, 4 objects/s recovering 2024-06-01T02:07:49.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:49 smithi116 ceph-mon[89502]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 93/726 objects degraded (12.810%), 19 pgs degraded) 2024-06-01T02:07:49.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:49 smithi116 ceph-mon[100606]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 93/726 objects degraded (12.810%), 19 pgs degraded) 2024-06-01T02:07:49.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:49 smithi161 ceph-mon[84406]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 93/726 objects degraded (12.810%), 19 pgs degraded) 2024-06-01T02:07:50.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:50 smithi116 ceph-mon[89502]: pgmap v127: 161 pgs: 1 peering, 160 active+clean; 585 KiB data, 141 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 245 B/s, 3 objects/s recovering 2024-06-01T02:07:50.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:50 smithi116 ceph-mon[100606]: pgmap v127: 161 pgs: 1 peering, 160 active+clean; 585 KiB data, 141 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 245 B/s, 3 objects/s recovering 2024-06-01T02:07:50.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:50 smithi161 ceph-mon[84406]: pgmap v127: 161 pgs: 1 peering, 160 active+clean; 585 KiB data, 141 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 245 B/s, 3 objects/s recovering 2024-06-01T02:07:52.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:52.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:52.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:51 smithi116 ceph-mon[100606]: pgmap v128: 161 pgs: 1 peering, 160 active+clean; 585 KiB data, 141 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 231 B/s, 2 objects/s recovering 2024-06-01T02:07:52.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:52.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:52.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:52.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:52.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:52.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:52.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:51 smithi116 ceph-mon[89502]: pgmap v128: 161 pgs: 1 peering, 160 active+clean; 585 KiB data, 141 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 231 B/s, 2 objects/s recovering 2024-06-01T02:07:52.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:52.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:52.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:52.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:52.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:52.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:52.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:51 smithi161 ceph-mon[84406]: pgmap v128: 161 pgs: 1 peering, 160 active+clean; 585 KiB data, 141 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 231 B/s, 2 objects/s recovering 2024-06-01T02:07:52.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:52.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:52.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:52.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:07:53.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:52 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:07:53.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:52 smithi116 ceph-mon[89502]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-06-01T02:07:53.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:52 smithi116 ceph-mon[89502]: Cluster is now healthy 2024-06-01T02:07:53.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:52 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:07:53.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:52 smithi116 ceph-mon[100606]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-06-01T02:07:53.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:52 smithi116 ceph-mon[100606]: Cluster is now healthy 2024-06-01T02:07:53.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:52 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:07:53.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:52 smithi161 ceph-mon[84406]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-06-01T02:07:53.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:52 smithi161 ceph-mon[84406]: Cluster is now healthy 2024-06-01T02:07:54.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:53 smithi116 ceph-mon[89502]: pgmap v129: 161 pgs: 161 active+clean; 585 KiB data, 141 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 41 op/s; 181 B/s, 2 objects/s recovering 2024-06-01T02:07:54.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:53 smithi116 ceph-mon[100606]: pgmap v129: 161 pgs: 161 active+clean; 585 KiB data, 141 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 41 op/s; 181 B/s, 2 objects/s recovering 2024-06-01T02:07:54.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:53 smithi161 ceph-mon[84406]: pgmap v129: 161 pgs: 161 active+clean; 585 KiB data, 141 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 41 op/s; 181 B/s, 2 objects/s recovering 2024-06-01T02:07:56.106 INFO:teuthology.orchestra.run.smithi116.stdout:true 2024-06-01T02:07:56.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:56 smithi116 ceph-mon[89502]: pgmap v130: 161 pgs: 161 active+clean; 585 KiB data, 141 MiB used, 715 GiB / 715 GiB avail; 22 KiB/s rd, 0 B/s wr, 35 op/s; 152 B/s, 1 objects/s recovering 2024-06-01T02:07:56.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:56 smithi116 ceph-mon[100606]: pgmap v130: 161 pgs: 161 active+clean; 585 KiB data, 141 MiB used, 715 GiB / 715 GiB avail; 22 KiB/s rd, 0 B/s wr, 35 op/s; 152 B/s, 1 objects/s recovering 2024-06-01T02:07:56.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:56 smithi161 ceph-mon[84406]: pgmap v130: 161 pgs: 161 active+clean; 585 KiB data, 141 MiB used, 715 GiB / 715 GiB avail; 22 KiB/s rd, 0 B/s wr, 35 op/s; 152 B/s, 1 objects/s recovering 2024-06-01T02:07:56.769 INFO:teuthology.orchestra.run.smithi116.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-01T02:07:56.769 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi116 smithi116 running 5s ago 16m - - 2024-06-01T02:07:56.770 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi161 smithi161 running 16s ago 16m - - 2024-06-01T02:07:56.770 INFO:teuthology.orchestra.run.smithi116.stdout:alertmanager.a smithi116 *:9093,9094 running (5m) 5s ago 11m 21.2M - 0.23.0 ba2b418f427c e0c297162ded 2024-06-01T02:07:56.770 INFO:teuthology.orchestra.run.smithi116.stdout:grafana.a smithi161 *:3000 running (5m) 16s ago 10m 52.9M - 8.3.5 dad864ee21e9 00b0a78ee88e 2024-06-01T02:07:56.770 INFO:teuthology.orchestra.run.smithi116.stdout:iscsi.foo.smithi116.gjuxzb smithi116 running (4m) 5s ago 9m 53.4M - 3.5 e1d6a67b021e 0e191fe3e3bb 2024-06-01T02:07:56.770 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.x smithi161 *:8443,9283,8765 running (4m) 16s ago 15m 524M - 19.0.0-2946-g3d89accf b045ac545f82 2d7fbb85e5f7 2024-06-01T02:07:56.770 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.y smithi116 *:8443,9283,8765 running (3m) 5s ago 17m 454M - 19.0.0-2946-g3d89accf b045ac545f82 bc9627c27a57 2024-06-01T02:07:56.770 INFO:teuthology.orchestra.run.smithi116.stdout:mon.a smithi116 running (3m) 5s ago 17m 57.0M 2048M 19.0.0-2946-g3d89accf b045ac545f82 b5e9b6f873ff 2024-06-01T02:07:56.770 INFO:teuthology.orchestra.run.smithi116.stdout:mon.b smithi161 running (2m) 16s ago 16m 38.1M 2048M 19.0.0-2946-g3d89accf b045ac545f82 aaa08c03cfc8 2024-06-01T02:07:56.770 INFO:teuthology.orchestra.run.smithi116.stdout:mon.c smithi116 running (2m) 5s ago 16m 43.2M 2048M 19.0.0-2946-g3d89accf b045ac545f82 bd0fb0e603fa 2024-06-01T02:07:56.771 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.a smithi116 *:9100 running (6m) 5s ago 11m 17.6M - 1.3.1 1dbe0e931976 e347748d17bb 2024-06-01T02:07:56.771 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.b smithi161 *:9100 running (5m) 16s ago 11m 17.4M - 1.3.1 1dbe0e931976 6e66182f5a1a 2024-06-01T02:07:56.771 INFO:teuthology.orchestra.run.smithi116.stdout:osd.0 smithi116 running (54s) 5s ago 15m 42.2M 4096M 19.0.0-2946-g3d89accf b045ac545f82 b4313d121b9d 2024-06-01T02:07:56.771 INFO:teuthology.orchestra.run.smithi116.stdout:osd.1 smithi116 running (19s) 5s ago 15m 44.7M 4096M 19.0.0-2946-g3d89accf b045ac545f82 dccdbeac5aae 2024-06-01T02:07:56.771 INFO:teuthology.orchestra.run.smithi116.stdout:osd.2 smithi116 running (14m) 5s ago 14m 56.8M 4096M 17.2.0 e1d6a67b021e 6aecbb10256b 2024-06-01T02:07:56.771 INFO:teuthology.orchestra.run.smithi116.stdout:osd.3 smithi116 running (14m) 5s ago 14m 59.0M 4096M 17.2.0 e1d6a67b021e a802290f5290 2024-06-01T02:07:56.771 INFO:teuthology.orchestra.run.smithi116.stdout:osd.4 smithi161 running (13m) 16s ago 13m 61.0M 4096M 17.2.0 e1d6a67b021e 70b3f89b0bd9 2024-06-01T02:07:56.771 INFO:teuthology.orchestra.run.smithi116.stdout:osd.5 smithi161 running (13m) 16s ago 13m 55.9M 4096M 17.2.0 e1d6a67b021e 68358b78a25d 2024-06-01T02:07:56.771 INFO:teuthology.orchestra.run.smithi116.stdout:osd.6 smithi161 running (13m) 16s ago 13m 55.3M 4096M 17.2.0 e1d6a67b021e 50abeb0551e7 2024-06-01T02:07:56.771 INFO:teuthology.orchestra.run.smithi116.stdout:osd.7 smithi161 running (12m) 16s ago 12m 58.7M 4096M 17.2.0 e1d6a67b021e 6004e6983735 2024-06-01T02:07:56.771 INFO:teuthology.orchestra.run.smithi116.stdout:prometheus.a smithi161 *:9095 running (4m) 16s ago 11m 74.8M - 2.33.4 514e6a882f6e e977f6ca1d73 2024-06-01T02:07:56.771 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi116.vzjrxj smithi116 *:8000 running (10m) 5s ago 10m 76.5M - 17.2.0 e1d6a67b021e df4fe3c1dbc3 2024-06-01T02:07:56.771 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi161.ashqxz smithi161 *:8000 running (10m) 16s ago 10m 77.5M - 17.2.0 e1d6a67b021e 3eecc1f3e2d3 2024-06-01T02:07:56.771 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi116.hokbmf smithi116 *:80 running (9m) 5s ago 9m 77.0M - 17.2.0 e1d6a67b021e 10950ed75dbd 2024-06-01T02:07:56.772 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi161.axjlzw smithi161 *:80 running (9m) 16s ago 9m 76.1M - 17.2.0 e1d6a67b021e 0b5d5ce47772 2024-06-01T02:07:57.051 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:07:56 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:07:56] "GET /metrics HTTP/1.1" 200 38295 "" "Prometheus/2.33.4" 2024-06-01T02:07:57.173 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:07:57.173 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": { 2024-06-01T02:07:57.173 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 3 2024-06-01T02:07:57.173 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:07:57.173 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": { 2024-06-01T02:07:57.173 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-01T02:07:57.174 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:07:57.174 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": { 2024-06-01T02:07:57.174 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 6, 2024-06-01T02:07:57.174 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-01T02:07:57.174 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:07:57.174 INFO:teuthology.orchestra.run.smithi116.stdout: "rgw": { 2024-06-01T02:07:57.174 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-06-01T02:07:57.174 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:07:57.174 INFO:teuthology.orchestra.run.smithi116.stdout: "overall": { 2024-06-01T02:07:57.174 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 10, 2024-06-01T02:07:57.174 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 7 2024-06-01T02:07:57.174 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T02:07:57.175 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:07:57.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:57 smithi161 ceph-mon[84406]: from='client.44298 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:07:57.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:57 smithi161 ceph-mon[84406]: from='client.64270 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:07:57.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:57 smithi161 ceph-mon[84406]: from='client.44310 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:07:57.507 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:07:57.507 INFO:teuthology.orchestra.run.smithi116.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-01T02:07:57.507 INFO:teuthology.orchestra.run.smithi116.stdout: "in_progress": true, 2024-06-01T02:07:57.507 INFO:teuthology.orchestra.run.smithi116.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-01T02:07:57.507 INFO:teuthology.orchestra.run.smithi116.stdout: "services_complete": [ 2024-06-01T02:07:57.507 INFO:teuthology.orchestra.run.smithi116.stdout: "mon", 2024-06-01T02:07:57.507 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr" 2024-06-01T02:07:57.507 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-06-01T02:07:57.507 INFO:teuthology.orchestra.run.smithi116.stdout: "progress": "7/23 daemons upgraded", 2024-06-01T02:07:57.507 INFO:teuthology.orchestra.run.smithi116.stdout: "message": "Currently upgrading osd daemons", 2024-06-01T02:07:57.507 INFO:teuthology.orchestra.run.smithi116.stdout: "is_paused": false 2024-06-01T02:07:57.508 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:07:57.508 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:57 smithi116 ceph-mon[100606]: from='client.44298 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:07:57.508 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:57 smithi116 ceph-mon[100606]: from='client.64270 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:07:57.508 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:57 smithi116 ceph-mon[100606]: from='client.44310 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:07:57.508 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:57 smithi116 ceph-mon[89502]: from='client.44298 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:07:57.508 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:57 smithi116 ceph-mon[89502]: from='client.64270 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:07:57.508 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:57 smithi116 ceph-mon[89502]: from='client.44310 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:07:57.897 INFO:teuthology.orchestra.run.smithi116.stdout:HEALTH_OK 2024-06-01T02:07:58.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:58 smithi116 ceph-mon[89502]: pgmap v131: 161 pgs: 161 active+clean; 585 KiB data, 151 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 0 B/s wr, 120 op/s 2024-06-01T02:07:58.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:58 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/827426342' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:58.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:58 smithi116 ceph-mon[89502]: from='client.44322 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:07:58.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:58 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/3671208573' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:07:58.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:58 smithi116 ceph-mon[100606]: pgmap v131: 161 pgs: 161 active+clean; 585 KiB data, 151 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 0 B/s wr, 120 op/s 2024-06-01T02:07:58.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:58 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/827426342' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:58.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:58 smithi116 ceph-mon[100606]: from='client.44322 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:07:58.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:58 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/3671208573' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:07:58.390 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:58 smithi161 ceph-mon[84406]: pgmap v131: 161 pgs: 161 active+clean; 585 KiB data, 151 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 0 B/s wr, 120 op/s 2024-06-01T02:07:58.391 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:58 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/827426342' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:07:58.391 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:58 smithi161 ceph-mon[84406]: from='client.44322 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:07:58.391 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:58 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/3671208573' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:07:59.320 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:59 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-06-01T02:07:59.320 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:59 smithi161 ceph-mon[84406]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-06-01T02:07:59.320 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:07:59 smithi161 ceph-mon[84406]: Upgrade: osd.2 is safe to restart 2024-06-01T02:07:59.356 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-06-01T02:07:59.356 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:59 smithi116 ceph-mon[89502]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-06-01T02:07:59.356 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:07:59 smithi116 ceph-mon[89502]: Upgrade: osd.2 is safe to restart 2024-06-01T02:07:59.357 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:59 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-06-01T02:07:59.357 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:59 smithi116 ceph-mon[100606]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-06-01T02:07:59.357 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:07:59 smithi116 ceph-mon[100606]: Upgrade: osd.2 is safe to restart 2024-06-01T02:08:00.379 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:00 smithi116 ceph-mon[100606]: pgmap v132: 161 pgs: 161 active+clean; 585 KiB data, 151 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s rd, 0 B/s wr, 127 op/s 2024-06-01T02:08:00.380 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:00 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:00.380 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:00 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:00.380 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:00 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:00.380 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:00 smithi116 ceph-mon[89502]: pgmap v132: 161 pgs: 161 active+clean; 585 KiB data, 151 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s rd, 0 B/s wr, 127 op/s 2024-06-01T02:08:00.380 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:00.380 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:00.380 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:00.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:00 smithi161 ceph-mon[84406]: pgmap v132: 161 pgs: 161 active+clean; 585 KiB data, 151 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s rd, 0 B/s wr, 127 op/s 2024-06-01T02:08:00.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:00 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:00.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:00 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:00.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:00 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:01.766 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:01 smithi116 ceph-mon[100606]: Upgrade: Updating osd.2 2024-06-01T02:08:01.766 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:01 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:01.766 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:01 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-01T02:08:01.766 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:01 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:01.766 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:01 smithi116 ceph-mon[100606]: Deploying daemon osd.2 on smithi116 2024-06-01T02:08:01.767 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:01 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:01.767 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:01 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:08:01.767 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:01 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:01.767 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:01 smithi116 ceph-mon[89502]: Upgrade: Updating osd.2 2024-06-01T02:08:01.767 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:01.767 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-01T02:08:01.767 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:01.767 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:01 smithi116 ceph-mon[89502]: Deploying daemon osd.2 on smithi116 2024-06-01T02:08:01.767 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:01.768 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:08:01.768 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:01.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:01 smithi161 ceph-mon[84406]: Upgrade: Updating osd.2 2024-06-01T02:08:01.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:01 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:01.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:01 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-01T02:08:01.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:01 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:01.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:01 smithi161 ceph-mon[84406]: Deploying daemon osd.2 on smithi116 2024-06-01T02:08:01.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:01 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:01.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:01 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:08:01.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:01 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:03.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:02 smithi161 ceph-mon[84406]: pgmap v133: 161 pgs: 161 active+clean; 585 KiB data, 151 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s rd, 0 B/s wr, 127 op/s 2024-06-01T02:08:03.205 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:02 smithi116 ceph-mon[89502]: pgmap v133: 161 pgs: 161 active+clean; 585 KiB data, 151 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s rd, 0 B/s wr, 127 op/s 2024-06-01T02:08:03.205 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:02 smithi116 ceph-mon[100606]: pgmap v133: 161 pgs: 161 active+clean; 585 KiB data, 151 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s rd, 0 B/s wr, 127 op/s 2024-06-01T02:08:04.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:03 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:08:04.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:03 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:08:04.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:03 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:08:04.793 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:04 smithi116 systemd[1]: Stopping Ceph osd.2 for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:08:05.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:04 smithi161 ceph-mon[84406]: pgmap v134: 161 pgs: 161 active+clean; 585 KiB data, 151 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 0 B/s wr, 120 op/s 2024-06-01T02:08:05.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:04 smithi161 ceph-mon[84406]: osd.2 marked itself down and dead 2024-06-01T02:08:05.205 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:04 smithi116 ceph-mon[89502]: pgmap v134: 161 pgs: 161 active+clean; 585 KiB data, 151 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 0 B/s wr, 120 op/s 2024-06-01T02:08:05.205 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:04 smithi116 ceph-mon[89502]: osd.2 marked itself down and dead 2024-06-01T02:08:05.206 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:04 smithi116 ceph-mon[100606]: pgmap v134: 161 pgs: 161 active+clean; 585 KiB data, 151 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 0 B/s wr, 120 op/s 2024-06-01T02:08:05.206 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:04 smithi116 ceph-mon[100606]: osd.2 marked itself down and dead 2024-06-01T02:08:05.206 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:04 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2[39044]: 2024-06-01T02:08:04.833+0000 7f5be1f0d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-01T02:08:05.206 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:04 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2[39044]: 2024-06-01T02:08:04.833+0000 7f5be1f0d700 -1 osd.2 138 *** Got signal Terminated *** 2024-06-01T02:08:05.206 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:04 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2[39044]: 2024-06-01T02:08:04.833+0000 7f5be1f0d700 -1 osd.2 138 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-01T02:08:05.543 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:05 smithi116 podman[122872]: 2024-06-01 02:08:05.43447413 +0000 UTC m=+0.715971077 container died 6aecbb10256b10dbbfb01980c5d1e92bf8d54ac72ee2f2735170c5a0ff1fab5b (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2, io.k8s.display-name=CentOS Stream 8, vcs-type=git, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, version=8, GIT_REPO=https://github.com/ceph/ceph-container.git, com.redhat.component=centos-stream-container, architecture=x86_64, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, vendor=Red Hat, Inc., maintainer=Guillaume Abrioux , name=centos-stream, io.buildah.version=1.19.8, io.openshift.expose-services=, build-date=2022-05-03T08:36:31.336870, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, RELEASE=HEAD, io.openshift.tags=base centos centos-stream, ceph=True, release=754, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, distribution-scope=public, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, GIT_CLEAN=True, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-17.2.0, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly.) 2024-06-01T02:08:06.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:05 smithi116 ceph-mon[89502]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:08:06.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:05 smithi116 ceph-mon[89502]: osdmap e139: 8 total, 7 up, 8 in 2024-06-01T02:08:06.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:05 smithi116 ceph-mon[100606]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:08:06.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:05 smithi116 ceph-mon[100606]: osdmap e139: 8 total, 7 up, 8 in 2024-06-01T02:08:06.043 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:05 smithi116 podman[122872]: 2024-06-01 02:08:05.825120342 +0000 UTC m=+1.106617286 container cleanup 6aecbb10256b10dbbfb01980c5d1e92bf8d54ac72ee2f2735170c5a0ff1fab5b (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2, io.openshift.expose-services=, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, distribution-scope=public, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, GIT_CLEAN=True, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., name=centos-stream, io.buildah.version=1.19.8, GIT_REPO=https://github.com/ceph/ceph-container.git, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, build-date=2022-05-03T08:36:31.336870, ceph=True, release=754, vcs-type=git, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, io.k8s.display-name=CentOS Stream 8, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, maintainer=Guillaume Abrioux , com.redhat.component=centos-stream-container, CEPH_POINT_RELEASE=-17.2.0, RELEASE=HEAD, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, io.openshift.tags=base centos centos-stream, version=8, vendor=Red Hat, Inc., GIT_BRANCH=HEAD, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, architecture=x86_64) 2024-06-01T02:08:06.044 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:05 smithi116 bash[122872]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2 2024-06-01T02:08:06.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:05 smithi161 ceph-mon[84406]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:08:06.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:05 smithi161 ceph-mon[84406]: osdmap e139: 8 total, 7 up, 8 in 2024-06-01T02:08:06.356 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:06 smithi116 podman[122999]: 2024-06-01 02:08:06.084936397 +0000 UTC m=+0.651134268 container remove 6aecbb10256b10dbbfb01980c5d1e92bf8d54ac72ee2f2735170c5a0ff1fab5b (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2, version=8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, com.redhat.component=centos-stream-container, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, distribution-scope=public, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, ceph=True, GIT_BRANCH=HEAD, io.k8s.display-name=CentOS Stream 8, io.openshift.tags=base centos centos-stream, GIT_CLEAN=True, build-date=2022-05-03T08:36:31.336870, RELEASE=HEAD, io.buildah.version=1.19.8, maintainer=Guillaume Abrioux , GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, io.openshift.expose-services=, vcs-type=git, release=754, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, architecture=x86_64, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., name=centos-stream, CEPH_POINT_RELEASE=-17.2.0, GIT_REPO=https://github.com/ceph/ceph-container.git, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, vendor=Red Hat, Inc.) 2024-06-01T02:08:07.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:06 smithi116 ceph-mon[100606]: pgmap v136: 161 pgs: 13 stale+active+clean, 148 active+clean; 585 KiB data, 151 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s rd, 0 B/s wr, 114 op/s 2024-06-01T02:08:07.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:06 smithi116 ceph-mon[100606]: osdmap e140: 8 total, 7 up, 8 in 2024-06-01T02:08:07.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:06 smithi116 ceph-mon[89502]: pgmap v136: 161 pgs: 13 stale+active+clean, 148 active+clean; 585 KiB data, 151 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s rd, 0 B/s wr, 114 op/s 2024-06-01T02:08:07.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:06 smithi116 ceph-mon[89502]: osdmap e140: 8 total, 7 up, 8 in 2024-06-01T02:08:07.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:06 smithi161 ceph-mon[84406]: pgmap v136: 161 pgs: 13 stale+active+clean, 148 active+clean; 585 KiB data, 151 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s rd, 0 B/s wr, 114 op/s 2024-06-01T02:08:07.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:06 smithi161 ceph-mon[84406]: osdmap e140: 8 total, 7 up, 8 in 2024-06-01T02:08:07.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:08:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:08:06] "GET /metrics HTTP/1.1" 200 38435 "" "Prometheus/2.33.4" 2024-06-01T02:08:07.437 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:07 smithi116 podman[123201]: 2024-06-01 02:08:07.164786428 +0000 UTC m=+0.112564329 container create 08fc2cb3d90ff60719d7660b1f325263e7dca062577401f9258ff1d5066fa901 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-06-01T02:08:07.437 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:07 smithi116 podman[123201]: 2024-06-01 02:08:07.072255681 +0000 UTC m=+0.020033591 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:08:07.437 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:07 smithi116 podman[123201]: 2024-06-01 02:08:07.3755039 +0000 UTC m=+0.323281781 container init 08fc2cb3d90ff60719d7660b1f325263e7dca062577401f9258ff1d5066fa901 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-06-01T02:08:07.437 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:07 smithi116 podman[123201]: 2024-06-01 02:08:07.379500602 +0000 UTC m=+0.327278482 container start 08fc2cb3d90ff60719d7660b1f325263e7dca062577401f9258ff1d5066fa901 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, ceph=True) 2024-06-01T02:08:07.438 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:07 smithi116 podman[123201]: 2024-06-01 02:08:07.442027705 +0000 UTC m=+0.389805594 container attach 08fc2cb3d90ff60719d7660b1f325263e7dca062577401f9258ff1d5066fa901 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-06-01T02:08:07.986 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:07 smithi116 podman[123201]: 2024-06-01 02:08:07.70663361 +0000 UTC m=+0.654411495 container died 08fc2cb3d90ff60719d7660b1f325263e7dca062577401f9258ff1d5066fa901 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-06-01T02:08:08.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:07 smithi116 ceph-mon[89502]: pgmap v138: 161 pgs: 12 active+undersized+degraded, 6 stale+active+clean, 23 active+undersized, 120 active+clean; 585 KiB data, 151 MiB used, 715 GiB / 715 GiB avail; 39/726 objects degraded (5.372%) 2024-06-01T02:08:08.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:07 smithi116 ceph-mon[89502]: Health check failed: Degraded data redundancy: 39/726 objects degraded (5.372%), 12 pgs degraded (PG_DEGRADED) 2024-06-01T02:08:08.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:07 smithi116 ceph-mon[100606]: pgmap v138: 161 pgs: 12 active+undersized+degraded, 6 stale+active+clean, 23 active+undersized, 120 active+clean; 585 KiB data, 151 MiB used, 715 GiB / 715 GiB avail; 39/726 objects degraded (5.372%) 2024-06-01T02:08:08.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:07 smithi116 ceph-mon[100606]: Health check failed: Degraded data redundancy: 39/726 objects degraded (5.372%), 12 pgs degraded (PG_DEGRADED) 2024-06-01T02:08:08.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:07 smithi161 ceph-mon[84406]: pgmap v138: 161 pgs: 12 active+undersized+degraded, 6 stale+active+clean, 23 active+undersized, 120 active+clean; 585 KiB data, 151 MiB used, 715 GiB / 715 GiB avail; 39/726 objects degraded (5.372%) 2024-06-01T02:08:08.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:07 smithi161 ceph-mon[84406]: Health check failed: Degraded data redundancy: 39/726 objects degraded (5.372%), 12 pgs degraded (PG_DEGRADED) 2024-06-01T02:08:08.793 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:08 smithi116 podman[123309]: 2024-06-01 02:08:08.538824479 +0000 UTC m=+0.825110145 container remove 08fc2cb3d90ff60719d7660b1f325263e7dca062577401f9258ff1d5066fa901 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD) 2024-06-01T02:08:08.793 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:08 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.2.service: Deactivated successfully. 2024-06-01T02:08:08.793 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:08 smithi116 systemd[1]: Stopped Ceph osd.2 for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:08:08.793 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:08 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.2.service: Consumed 7.269s CPU time. 2024-06-01T02:08:08.793 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:08 smithi116 systemd[1]: Starting Ceph osd.2 for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:08:10.276 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:10 smithi116 ceph-mon[89502]: pgmap v139: 161 pgs: 16 active+undersized+degraded, 35 active+undersized, 110 active+clean; 585 KiB data, 151 MiB used, 715 GiB / 715 GiB avail; 68/726 objects degraded (9.366%) 2024-06-01T02:08:10.277 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:10 smithi116 ceph-mon[100606]: pgmap v139: 161 pgs: 16 active+undersized+degraded, 35 active+undersized, 110 active+clean; 585 KiB data, 151 MiB used, 715 GiB / 715 GiB avail; 68/726 objects degraded (9.366%) 2024-06-01T02:08:10.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:10 smithi161 ceph-mon[84406]: pgmap v139: 161 pgs: 16 active+undersized+degraded, 35 active+undersized, 110 active+clean; 585 KiB data, 151 MiB used, 715 GiB / 715 GiB avail; 68/726 objects degraded (9.366%) 2024-06-01T02:08:10.543 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:10 smithi116 podman[123522]: 2024-06-01 02:08:10.281370111 +0000 UTC m=+0.739763261 container create 676c1050a19551072509eb3b8b21b4eec05639fbd9f6536d28de4e3fe205d266 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2-activate, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-06-01T02:08:10.543 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:10 smithi116 podman[123522]: 2024-06-01 02:08:10.189571662 +0000 UTC m=+0.647964780 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:08:10.543 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:10 smithi116 podman[123522]: 2024-06-01 02:08:10.517322833 +0000 UTC m=+0.975715938 container init 676c1050a19551072509eb3b8b21b4eec05639fbd9f6536d28de4e3fe205d266 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2-activate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc) 2024-06-01T02:08:10.543 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:10 smithi116 podman[123522]: 2024-06-01 02:08:10.520697708 +0000 UTC m=+0.979090832 container start 676c1050a19551072509eb3b8b21b4eec05639fbd9f6536d28de4e3fe205d266 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2-activate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-01T02:08:10.921 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:10 smithi116 podman[123522]: 2024-06-01 02:08:10.608876278 +0000 UTC m=+1.067269391 container attach 676c1050a19551072509eb3b8b21b4eec05639fbd9f6536d28de4e3fe205d266 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2-activate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-01T02:08:11.293 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:10 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2-activate[123593]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-06-01T02:08:11.293 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:10 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2-activate[123593]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-06-01T02:08:11.293 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:10 smithi116 bash[123522]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-06-01T02:08:11.293 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:10 smithi116 bash[123522]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-06-01T02:08:11.293 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:10 smithi116 bash[123522]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-06-01T02:08:11.293 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:10 smithi116 bash[123522]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-06-01T02:08:11.293 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:10 smithi116 bash[123522]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-06-01T02:08:11.293 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:10 smithi116 bash[123522]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-06-01T02:08:11.294 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:10 smithi116 bash[123522]: --> ceph-volume raw activate successful for osd ID: 2 2024-06-01T02:08:11.294 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:10 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2-activate[123593]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-06-01T02:08:11.294 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:10 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2-activate[123593]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-06-01T02:08:11.294 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:10 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2-activate[123593]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-06-01T02:08:11.294 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:10 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2-activate[123593]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-06-01T02:08:11.294 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:10 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2-activate[123593]: --> ceph-volume raw activate successful for osd ID: 2 2024-06-01T02:08:11.645 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:11 smithi116 podman[123522]: 2024-06-01 02:08:11.321040452 +0000 UTC m=+1.779433566 container died 676c1050a19551072509eb3b8b21b4eec05639fbd9f6536d28de4e3fe205d266 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2-activate, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc) 2024-06-01T02:08:12.292 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:12 smithi116 podman[123752]: 2024-06-01 02:08:12.167788614 +0000 UTC m=+1.169881616 container remove 676c1050a19551072509eb3b8b21b4eec05639fbd9f6536d28de4e3fe205d266 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2-activate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-06-01T02:08:12.599 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:12 smithi116 ceph-mon[89502]: pgmap v140: 161 pgs: 16 active+undersized+degraded, 35 active+undersized, 110 active+clean; 585 KiB data, 151 MiB used, 715 GiB / 715 GiB avail; 68/726 objects degraded (9.366%) 2024-06-01T02:08:12.599 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:12 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:12.600 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:12 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:12.600 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:12 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:12.600 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:12 smithi116 ceph-mon[100606]: pgmap v140: 161 pgs: 16 active+undersized+degraded, 35 active+undersized, 110 active+clean; 585 KiB data, 151 MiB used, 715 GiB / 715 GiB avail; 68/726 objects degraded (9.366%) 2024-06-01T02:08:12.600 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:12 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:12.600 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:12 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:12.600 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:12 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:12.601 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:12 smithi116 podman[123875]: 2024-06-01 02:08:12.284364758 +0000 UTC m=+0.020088679 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:08:12.601 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:12 smithi116 podman[123875]: 2024-06-01 02:08:12.385581283 +0000 UTC m=+0.121305175 container create 1ad95efda4043f841727af00c02ba41530144fe0b34aed28f3efbc14296ed566 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-01T02:08:12.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:12 smithi161 ceph-mon[84406]: pgmap v140: 161 pgs: 16 active+undersized+degraded, 35 active+undersized, 110 active+clean; 585 KiB data, 151 MiB used, 715 GiB / 715 GiB avail; 68/726 objects degraded (9.366%) 2024-06-01T02:08:12.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:12 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:12.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:12 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:12.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:12 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:12.910 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:12 smithi116 podman[123875]: 2024-06-01 02:08:12.604724576 +0000 UTC m=+0.340448485 container init 1ad95efda4043f841727af00c02ba41530144fe0b34aed28f3efbc14296ed566 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2, org.label-schema.build-date=20240528, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-01T02:08:12.910 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:12 smithi116 podman[123875]: 2024-06-01 02:08:12.608152678 +0000 UTC m=+0.343876572 container start 1ad95efda4043f841727af00c02ba41530144fe0b34aed28f3efbc14296ed566 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True) 2024-06-01T02:08:12.910 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:12 smithi116 ceph-osd[123925]: -- 172.21.15.116:0/440243528 <== mon.1 v2:172.21.15.116:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55adf49b7680 con 0x55adf4980c00 2024-06-01T02:08:12.910 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:12 smithi116 bash[123875]: 1ad95efda4043f841727af00c02ba41530144fe0b34aed28f3efbc14296ed566 2024-06-01T02:08:12.910 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:12 smithi116 systemd[1]: Started Ceph osd.2 for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:08:13.292 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2[123906]: 2024-06-01T02:08:13.150+0000 7fae477ad7c0 -1 Falling back to public interface 2024-06-01T02:08:13.944 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:08:13.574+0000 7fcedaabb700 -1 mgr.server reply reply (16) Device or resource busy unsafe to stop osd(s) at this time (17 PGs are or would become offline) 2024-06-01T02:08:13.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:08:13.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:13.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:13.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:13.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:13.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:13.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:13.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:13.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:13.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:13.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:13.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:13.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:13.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:13.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:13.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:13.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:13.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:13.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:13.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:13.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:13.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:13.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:13.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:13.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:13.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:13.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:13.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:13.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:13.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:13.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-06-01T02:08:14.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:08:14.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:14.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:14.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:14.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:14.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:14.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:14.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:14.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:14.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:14.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:14.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:14.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:14.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:14.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:14.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:14.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:14.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:14.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:14.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:14.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:14.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:14.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:14.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:14.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:14.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:14.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:14.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:14.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:08:14.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:14.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:14.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:14.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:14.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:14.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:14.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:14.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:14.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:14.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:14.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:14.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:14.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:14.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:14.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:14.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:14.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:14.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:14.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:14.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:14.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:14.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:14.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:14.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:14.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:14.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:14.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:14.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:14.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:14.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:14.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:14.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:14.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:14.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:14.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:14.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:14.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:14.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:14.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:14.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:14.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:14.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:14.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:14.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-06-01T02:08:14.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:14.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:14.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:14.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:14.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:14.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:14.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:14.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:14.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:14.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:14.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:14.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:14.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:14.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:14.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:14.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:14.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-06-01T02:08:15.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:14 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:15.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:14 smithi116 ceph-mon[100606]: pgmap v141: 161 pgs: 16 active+undersized+degraded, 35 active+undersized, 110 active+clean; 585 KiB data, 160 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s rd, 0 B/s wr, 138 op/s; 68/726 objects degraded (9.366%) 2024-06-01T02:08:15.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:14 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:15.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:14 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:15.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:14 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:15.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:14 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:15.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:14 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:15.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:14 smithi116 ceph-mon[100606]: pgmap v142: 161 pgs: 16 active+undersized+degraded, 35 active+undersized, 110 active+clean; 585 KiB data, 160 MiB used, 715 GiB / 715 GiB avail; 79 KiB/s rd, 0 B/s wr, 131 op/s; 68/726 objects degraded (9.366%) 2024-06-01T02:08:15.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:14 smithi116 ceph-mon[100606]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-06-01T02:08:15.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:14 smithi116 ceph-mon[100606]: Upgrade: unsafe to stop osd(s) at this time (17 PGs are or would become offline) 2024-06-01T02:08:15.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:14 smithi116 ceph-mon[100606]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-06-01T02:08:15.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:14 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:15.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:14 smithi116 ceph-mon[89502]: pgmap v141: 161 pgs: 16 active+undersized+degraded, 35 active+undersized, 110 active+clean; 585 KiB data, 160 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s rd, 0 B/s wr, 138 op/s; 68/726 objects degraded (9.366%) 2024-06-01T02:08:15.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:14 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:15.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:14 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:15.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:14 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:15.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:14 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:15.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:14 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:15.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:14 smithi116 ceph-mon[89502]: pgmap v142: 161 pgs: 16 active+undersized+degraded, 35 active+undersized, 110 active+clean; 585 KiB data, 160 MiB used, 715 GiB / 715 GiB avail; 79 KiB/s rd, 0 B/s wr, 131 op/s; 68/726 objects degraded (9.366%) 2024-06-01T02:08:15.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:14 smithi116 ceph-mon[89502]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-06-01T02:08:15.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:14 smithi116 ceph-mon[89502]: Upgrade: unsafe to stop osd(s) at this time (17 PGs are or would become offline) 2024-06-01T02:08:15.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:14 smithi116 ceph-mon[89502]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-06-01T02:08:15.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:14 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:15.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:14 smithi161 ceph-mon[84406]: pgmap v141: 161 pgs: 16 active+undersized+degraded, 35 active+undersized, 110 active+clean; 585 KiB data, 160 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s rd, 0 B/s wr, 138 op/s; 68/726 objects degraded (9.366%) 2024-06-01T02:08:15.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:14 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:15.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:14 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:15.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:14 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:15.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:14 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:15.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:14 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:15.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:14 smithi161 ceph-mon[84406]: pgmap v142: 161 pgs: 16 active+undersized+degraded, 35 active+undersized, 110 active+clean; 585 KiB data, 160 MiB used, 715 GiB / 715 GiB avail; 79 KiB/s rd, 0 B/s wr, 131 op/s; 68/726 objects degraded (9.366%) 2024-06-01T02:08:15.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:14 smithi161 ceph-mon[84406]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-06-01T02:08:15.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:14 smithi161 ceph-mon[84406]: Upgrade: unsafe to stop osd(s) at this time (17 PGs are or would become offline) 2024-06-01T02:08:15.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:14 smithi161 ceph-mon[84406]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-06-01T02:08:16.043 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:15 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2[123906]: 2024-06-01T02:08:15.868+0000 7fae477ad7c0 -1 osd.2 0 read_superblock omap replica is missing. 2024-06-01T02:08:16.043 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:15 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2[123906]: 2024-06-01T02:08:15.892+0000 7fae477ad7c0 -1 osd.2 138 log_to_monitors true 2024-06-01T02:08:17.141 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:08:16 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2[123906]: 2024-06-01T02:08:16.878+0000 7fae38c04700 -1 osd.2 138 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-01T02:08:17.142 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:16 smithi116 ceph-mon[100606]: pgmap v143: 161 pgs: 16 active+undersized+degraded, 35 active+undersized, 110 active+clean; 585 KiB data, 160 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s rd, 0 B/s wr, 116 op/s; 68/726 objects degraded (9.366%) 2024-06-01T02:08:17.142 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:17.142 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:08:17.142 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:16 smithi116 ceph-mon[100606]: from='osd.2 [v2:172.21.15.116:6816/2234868031,v1:172.21.15.116:6817/2234868031]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-01T02:08:17.142 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:16 smithi116 ceph-mon[100606]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-01T02:08:17.142 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:16 smithi116 ceph-mon[100606]: Health check update: Degraded data redundancy: 68/726 objects degraded (9.366%), 16 pgs degraded (PG_DEGRADED) 2024-06-01T02:08:17.142 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:16 smithi116 ceph-mon[89502]: pgmap v143: 161 pgs: 16 active+undersized+degraded, 35 active+undersized, 110 active+clean; 585 KiB data, 160 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s rd, 0 B/s wr, 116 op/s; 68/726 objects degraded (9.366%) 2024-06-01T02:08:17.142 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:17.142 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:08:17.142 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:16 smithi116 ceph-mon[89502]: from='osd.2 [v2:172.21.15.116:6816/2234868031,v1:172.21.15.116:6817/2234868031]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-01T02:08:17.143 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:16 smithi116 ceph-mon[89502]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-01T02:08:17.143 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:16 smithi116 ceph-mon[89502]: Health check update: Degraded data redundancy: 68/726 objects degraded (9.366%), 16 pgs degraded (PG_DEGRADED) 2024-06-01T02:08:17.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:08:16 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:08:16] "GET /metrics HTTP/1.1" 200 38434 "" "Prometheus/2.33.4" 2024-06-01T02:08:17.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:16 smithi161 ceph-mon[84406]: pgmap v143: 161 pgs: 16 active+undersized+degraded, 35 active+undersized, 110 active+clean; 585 KiB data, 160 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s rd, 0 B/s wr, 116 op/s; 68/726 objects degraded (9.366%) 2024-06-01T02:08:17.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:17.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:08:17.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:16 smithi161 ceph-mon[84406]: from='osd.2 [v2:172.21.15.116:6816/2234868031,v1:172.21.15.116:6817/2234868031]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-01T02:08:17.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:16 smithi161 ceph-mon[84406]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-01T02:08:17.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:16 smithi161 ceph-mon[84406]: Health check update: Degraded data redundancy: 68/726 objects degraded (9.366%), 16 pgs degraded (PG_DEGRADED) 2024-06-01T02:08:18.144 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:17 smithi116 ceph-mon[89502]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-01T02:08:18.145 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:17 smithi116 ceph-mon[89502]: osdmap e141: 8 total, 7 up, 8 in 2024-06-01T02:08:18.145 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:17 smithi116 ceph-mon[89502]: from='osd.2 [v2:172.21.15.116:6816/2234868031,v1:172.21.15.116:6817/2234868031]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T02:08:18.145 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:17 smithi116 ceph-mon[89502]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T02:08:18.145 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:17 smithi116 ceph-mon[100606]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-01T02:08:18.146 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:17 smithi116 ceph-mon[100606]: osdmap e141: 8 total, 7 up, 8 in 2024-06-01T02:08:18.146 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:17 smithi116 ceph-mon[100606]: from='osd.2 [v2:172.21.15.116:6816/2234868031,v1:172.21.15.116:6817/2234868031]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T02:08:18.146 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:17 smithi116 ceph-mon[100606]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T02:08:18.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:17 smithi161 ceph-mon[84406]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-01T02:08:18.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:17 smithi161 ceph-mon[84406]: osdmap e141: 8 total, 7 up, 8 in 2024-06-01T02:08:18.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:17 smithi161 ceph-mon[84406]: from='osd.2 [v2:172.21.15.116:6816/2234868031,v1:172.21.15.116:6817/2234868031]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T02:08:18.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:17 smithi161 ceph-mon[84406]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T02:08:19.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:18 smithi116 ceph-mon[89502]: pgmap v145: 161 pgs: 16 active+undersized+degraded, 35 active+undersized, 110 active+clean; 585 KiB data, 183 MiB used, 715 GiB / 715 GiB avail; 102 KiB/s rd, 0 B/s wr, 169 op/s; 68/726 objects degraded (9.366%) 2024-06-01T02:08:19.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:18 smithi116 ceph-mon[89502]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:08:19.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:18 smithi116 ceph-mon[89502]: osd.2 [v2:172.21.15.116:6816/2234868031,v1:172.21.15.116:6817/2234868031] boot 2024-06-01T02:08:19.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:18 smithi116 ceph-mon[89502]: osdmap e142: 8 total, 8 up, 8 in 2024-06-01T02:08:19.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T02:08:19.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:18 smithi116 ceph-mon[89502]: osdmap e143: 8 total, 8 up, 8 in 2024-06-01T02:08:19.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:18 smithi116 ceph-mon[100606]: pgmap v145: 161 pgs: 16 active+undersized+degraded, 35 active+undersized, 110 active+clean; 585 KiB data, 183 MiB used, 715 GiB / 715 GiB avail; 102 KiB/s rd, 0 B/s wr, 169 op/s; 68/726 objects degraded (9.366%) 2024-06-01T02:08:19.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:18 smithi116 ceph-mon[100606]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:08:19.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:18 smithi116 ceph-mon[100606]: osd.2 [v2:172.21.15.116:6816/2234868031,v1:172.21.15.116:6817/2234868031] boot 2024-06-01T02:08:19.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:18 smithi116 ceph-mon[100606]: osdmap e142: 8 total, 8 up, 8 in 2024-06-01T02:08:19.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T02:08:19.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:18 smithi116 ceph-mon[100606]: osdmap e143: 8 total, 8 up, 8 in 2024-06-01T02:08:19.138 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:18 smithi161 ceph-mon[84406]: pgmap v145: 161 pgs: 16 active+undersized+degraded, 35 active+undersized, 110 active+clean; 585 KiB data, 183 MiB used, 715 GiB / 715 GiB avail; 102 KiB/s rd, 0 B/s wr, 169 op/s; 68/726 objects degraded (9.366%) 2024-06-01T02:08:19.138 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:18 smithi161 ceph-mon[84406]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:08:19.138 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:18 smithi161 ceph-mon[84406]: osd.2 [v2:172.21.15.116:6816/2234868031,v1:172.21.15.116:6817/2234868031] boot 2024-06-01T02:08:19.139 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:18 smithi161 ceph-mon[84406]: osdmap e142: 8 total, 8 up, 8 in 2024-06-01T02:08:19.139 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-01T02:08:19.139 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:18 smithi161 ceph-mon[84406]: osdmap e143: 8 total, 8 up, 8 in 2024-06-01T02:08:21.018 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:20 smithi116 ceph-mon[100606]: pgmap v148: 161 pgs: 16 active+undersized+degraded, 35 active+undersized, 110 active+clean; 585 KiB data, 183 MiB used, 715 GiB / 715 GiB avail; 32 KiB/s rd, 0 B/s wr, 52 op/s; 68/726 objects degraded (9.366%) 2024-06-01T02:08:21.018 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:21.019 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:21.019 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:21.019 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:20 smithi116 ceph-mon[89502]: pgmap v148: 161 pgs: 16 active+undersized+degraded, 35 active+undersized, 110 active+clean; 585 KiB data, 183 MiB used, 715 GiB / 715 GiB avail; 32 KiB/s rd, 0 B/s wr, 52 op/s; 68/726 objects degraded (9.366%) 2024-06-01T02:08:21.019 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:21.019 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:21.019 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:21.191 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:20 smithi161 ceph-mon[84406]: pgmap v148: 161 pgs: 16 active+undersized+degraded, 35 active+undersized, 110 active+clean; 585 KiB data, 183 MiB used, 715 GiB / 715 GiB avail; 32 KiB/s rd, 0 B/s wr, 52 op/s; 68/726 objects degraded (9.366%) 2024-06-01T02:08:21.191 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:21.191 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:21.192 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:22.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:21 smithi161 ceph-mon[84406]: pgmap v149: 161 pgs: 16 active+undersized+degraded, 35 active+undersized, 110 active+clean; 585 KiB data, 183 MiB used, 715 GiB / 715 GiB avail; 41 KiB/s rd, 0 B/s wr, 67 op/s; 68/726 objects degraded (9.366%) 2024-06-01T02:08:22.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:21 smithi161 ceph-mon[84406]: pgmap v150: 161 pgs: 16 active+undersized+degraded, 35 active+undersized, 110 active+clean; 585 KiB data, 183 MiB used, 715 GiB / 715 GiB avail; 68/726 objects degraded (9.366%) 2024-06-01T02:08:22.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:22.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:22.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:22.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:21 smithi116 ceph-mon[89502]: pgmap v149: 161 pgs: 16 active+undersized+degraded, 35 active+undersized, 110 active+clean; 585 KiB data, 183 MiB used, 715 GiB / 715 GiB avail; 41 KiB/s rd, 0 B/s wr, 67 op/s; 68/726 objects degraded (9.366%) 2024-06-01T02:08:22.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:21 smithi116 ceph-mon[89502]: pgmap v150: 161 pgs: 16 active+undersized+degraded, 35 active+undersized, 110 active+clean; 585 KiB data, 183 MiB used, 715 GiB / 715 GiB avail; 68/726 objects degraded (9.366%) 2024-06-01T02:08:22.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:22.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:22.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:22.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:21 smithi116 ceph-mon[100606]: pgmap v149: 161 pgs: 16 active+undersized+degraded, 35 active+undersized, 110 active+clean; 585 KiB data, 183 MiB used, 715 GiB / 715 GiB avail; 41 KiB/s rd, 0 B/s wr, 67 op/s; 68/726 objects degraded (9.366%) 2024-06-01T02:08:22.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:21 smithi116 ceph-mon[100606]: pgmap v150: 161 pgs: 16 active+undersized+degraded, 35 active+undersized, 110 active+clean; 585 KiB data, 183 MiB used, 715 GiB / 715 GiB avail; 68/726 objects degraded (9.366%) 2024-06-01T02:08:22.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:22.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:22.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:23.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:22 smithi161 ceph-mon[84406]: pgmap v151: 161 pgs: 16 active+undersized+degraded, 35 active+undersized, 110 active+clean; 585 KiB data, 183 MiB used, 715 GiB / 715 GiB avail; 68/726 objects degraded (9.366%) 2024-06-01T02:08:23.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:22 smithi161 ceph-mon[84406]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-06-01T02:08:23.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:22 smithi116 ceph-mon[100606]: pgmap v151: 161 pgs: 16 active+undersized+degraded, 35 active+undersized, 110 active+clean; 585 KiB data, 183 MiB used, 715 GiB / 715 GiB avail; 68/726 objects degraded (9.366%) 2024-06-01T02:08:23.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:22 smithi116 ceph-mon[100606]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-06-01T02:08:23.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:22 smithi116 ceph-mon[89502]: pgmap v151: 161 pgs: 16 active+undersized+degraded, 35 active+undersized, 110 active+clean; 585 KiB data, 183 MiB used, 715 GiB / 715 GiB avail; 68/726 objects degraded (9.366%) 2024-06-01T02:08:23.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:22 smithi116 ceph-mon[89502]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-06-01T02:08:24.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:23 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:08:24.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:23 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:08:24.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:23 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:08:25.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:24 smithi161 ceph-mon[84406]: pgmap v152: 161 pgs: 161 active+clean; 585 KiB data, 184 MiB used, 715 GiB / 715 GiB avail; 917 B/s rd, 0 op/s; 0 B/s, 6 objects/s recovering 2024-06-01T02:08:25.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:24 smithi161 ceph-mon[84406]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 68/726 objects degraded (9.366%), 16 pgs degraded) 2024-06-01T02:08:25.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:24 smithi161 ceph-mon[84406]: Cluster is now healthy 2024-06-01T02:08:25.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:24 smithi116 ceph-mon[100606]: pgmap v152: 161 pgs: 161 active+clean; 585 KiB data, 184 MiB used, 715 GiB / 715 GiB avail; 917 B/s rd, 0 op/s; 0 B/s, 6 objects/s recovering 2024-06-01T02:08:25.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:24 smithi116 ceph-mon[100606]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 68/726 objects degraded (9.366%), 16 pgs degraded) 2024-06-01T02:08:25.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:24 smithi116 ceph-mon[100606]: Cluster is now healthy 2024-06-01T02:08:25.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:24 smithi116 ceph-mon[89502]: pgmap v152: 161 pgs: 161 active+clean; 585 KiB data, 184 MiB used, 715 GiB / 715 GiB avail; 917 B/s rd, 0 op/s; 0 B/s, 6 objects/s recovering 2024-06-01T02:08:25.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:24 smithi116 ceph-mon[89502]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 68/726 objects degraded (9.366%), 16 pgs degraded) 2024-06-01T02:08:25.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:24 smithi116 ceph-mon[89502]: Cluster is now healthy 2024-06-01T02:08:27.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:08:26 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:08:26] "GET /metrics HTTP/1.1" 200 38434 "" "Prometheus/2.33.4" 2024-06-01T02:08:27.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:26 smithi161 ceph-mon[84406]: pgmap v153: 161 pgs: 161 active+clean; 585 KiB data, 184 MiB used, 715 GiB / 715 GiB avail; 777 B/s rd, 0 op/s; 0 B/s, 5 objects/s recovering 2024-06-01T02:08:27.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:26 smithi116 ceph-mon[89502]: pgmap v153: 161 pgs: 161 active+clean; 585 KiB data, 184 MiB used, 715 GiB / 715 GiB avail; 777 B/s rd, 0 op/s; 0 B/s, 5 objects/s recovering 2024-06-01T02:08:27.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:26 smithi116 ceph-mon[100606]: pgmap v153: 161 pgs: 161 active+clean; 585 KiB data, 184 MiB used, 715 GiB / 715 GiB avail; 777 B/s rd, 0 op/s; 0 B/s, 5 objects/s recovering 2024-06-01T02:08:28.255 INFO:teuthology.orchestra.run.smithi116.stdout:true 2024-06-01T02:08:28.916 INFO:teuthology.orchestra.run.smithi116.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-01T02:08:28.917 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi116 smithi116 running 7s ago 17m - - 2024-06-01T02:08:28.917 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi161 smithi161 running 8s ago 16m - - 2024-06-01T02:08:28.917 INFO:teuthology.orchestra.run.smithi116.stdout:alertmanager.a smithi116 *:9093,9094 running (6m) 7s ago 11m 21.2M - 0.23.0 ba2b418f427c e0c297162ded 2024-06-01T02:08:28.917 INFO:teuthology.orchestra.run.smithi116.stdout:grafana.a smithi161 *:3000 running (6m) 8s ago 11m 53.0M - 8.3.5 dad864ee21e9 00b0a78ee88e 2024-06-01T02:08:28.917 INFO:teuthology.orchestra.run.smithi116.stdout:iscsi.foo.smithi116.gjuxzb smithi116 running (5m) 7s ago 10m 53.5M - 3.5 e1d6a67b021e 0e191fe3e3bb 2024-06-01T02:08:28.917 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.x smithi161 *:8443,9283,8765 running (5m) 8s ago 16m 525M - 19.0.0-2946-g3d89accf b045ac545f82 2d7fbb85e5f7 2024-06-01T02:08:28.917 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.y smithi116 *:8443,9283,8765 running (4m) 7s ago 17m 454M - 19.0.0-2946-g3d89accf b045ac545f82 bc9627c27a57 2024-06-01T02:08:28.917 INFO:teuthology.orchestra.run.smithi116.stdout:mon.a smithi116 running (3m) 7s ago 18m 57.8M 2048M 19.0.0-2946-g3d89accf b045ac545f82 b5e9b6f873ff 2024-06-01T02:08:28.917 INFO:teuthology.orchestra.run.smithi116.stdout:mon.b smithi161 running (2m) 8s ago 16m 38.4M 2048M 19.0.0-2946-g3d89accf b045ac545f82 aaa08c03cfc8 2024-06-01T02:08:28.917 INFO:teuthology.orchestra.run.smithi116.stdout:mon.c smithi116 running (2m) 7s ago 16m 43.3M 2048M 19.0.0-2946-g3d89accf b045ac545f82 bd0fb0e603fa 2024-06-01T02:08:28.917 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.a smithi116 *:9100 running (6m) 7s ago 12m 20.1M - 1.3.1 1dbe0e931976 e347748d17bb 2024-06-01T02:08:28.917 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.b smithi161 *:9100 running (6m) 8s ago 12m 17.6M - 1.3.1 1dbe0e931976 6e66182f5a1a 2024-06-01T02:08:28.918 INFO:teuthology.orchestra.run.smithi116.stdout:osd.0 smithi116 running (86s) 7s ago 15m 45.1M 4096M 19.0.0-2946-g3d89accf b045ac545f82 b4313d121b9d 2024-06-01T02:08:28.918 INFO:teuthology.orchestra.run.smithi116.stdout:osd.1 smithi116 running (51s) 7s ago 15m 47.9M 4096M 19.0.0-2946-g3d89accf b045ac545f82 dccdbeac5aae 2024-06-01T02:08:28.918 INFO:teuthology.orchestra.run.smithi116.stdout:osd.2 smithi116 running (16s) 7s ago 15m 12.8M 4096M 19.0.0-2946-g3d89accf b045ac545f82 1ad95efda404 2024-06-01T02:08:28.918 INFO:teuthology.orchestra.run.smithi116.stdout:osd.3 smithi116 running (14m) 7s ago 14m 61.7M 4096M 17.2.0 e1d6a67b021e a802290f5290 2024-06-01T02:08:28.918 INFO:teuthology.orchestra.run.smithi116.stdout:osd.4 smithi161 running (14m) 8s ago 14m 63.0M 4096M 17.2.0 e1d6a67b021e 70b3f89b0bd9 2024-06-01T02:08:28.918 INFO:teuthology.orchestra.run.smithi116.stdout:osd.5 smithi161 running (13m) 8s ago 13m 58.3M 4096M 17.2.0 e1d6a67b021e 68358b78a25d 2024-06-01T02:08:28.918 INFO:teuthology.orchestra.run.smithi116.stdout:osd.6 smithi161 running (13m) 8s ago 13m 58.3M 4096M 17.2.0 e1d6a67b021e 50abeb0551e7 2024-06-01T02:08:28.918 INFO:teuthology.orchestra.run.smithi116.stdout:osd.7 smithi161 running (13m) 8s ago 13m 62.7M 4096M 17.2.0 e1d6a67b021e 6004e6983735 2024-06-01T02:08:28.918 INFO:teuthology.orchestra.run.smithi116.stdout:prometheus.a smithi161 *:9095 running (5m) 8s ago 12m 76.9M - 2.33.4 514e6a882f6e e977f6ca1d73 2024-06-01T02:08:28.918 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi116.vzjrxj smithi116 *:8000 running (10m) 7s ago 10m 76.7M - 17.2.0 e1d6a67b021e df4fe3c1dbc3 2024-06-01T02:08:28.918 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi161.ashqxz smithi161 *:8000 running (10m) 8s ago 10m 77.7M - 17.2.0 e1d6a67b021e 3eecc1f3e2d3 2024-06-01T02:08:28.918 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi116.hokbmf smithi116 *:80 running (10m) 7s ago 10m 77.1M - 17.2.0 e1d6a67b021e 10950ed75dbd 2024-06-01T02:08:28.918 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi161.axjlzw smithi161 *:80 running (10m) 8s ago 10m 76.4M - 17.2.0 e1d6a67b021e 0b5d5ce47772 2024-06-01T02:08:29.191 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:28 smithi116 ceph-mon[100606]: pgmap v154: 161 pgs: 161 active+clean; 585 KiB data, 184 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 0 B/s, 4 objects/s recovering 2024-06-01T02:08:29.191 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-06-01T02:08:29.191 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:28 smithi116 ceph-mon[89502]: pgmap v154: 161 pgs: 161 active+clean; 585 KiB data, 184 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 0 B/s, 4 objects/s recovering 2024-06-01T02:08:29.191 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-06-01T02:08:29.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:28 smithi161 ceph-mon[84406]: pgmap v154: 161 pgs: 161 active+clean; 585 KiB data, 184 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 0 B/s, 4 objects/s recovering 2024-06-01T02:08:29.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-06-01T02:08:29.331 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:08:29.331 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": { 2024-06-01T02:08:29.331 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 3 2024-06-01T02:08:29.331 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:08:29.331 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": { 2024-06-01T02:08:29.331 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-01T02:08:29.332 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:08:29.332 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": { 2024-06-01T02:08:29.332 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 5, 2024-06-01T02:08:29.332 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 3 2024-06-01T02:08:29.332 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:08:29.332 INFO:teuthology.orchestra.run.smithi116.stdout: "rgw": { 2024-06-01T02:08:29.332 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-06-01T02:08:29.332 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:08:29.332 INFO:teuthology.orchestra.run.smithi116.stdout: "overall": { 2024-06-01T02:08:29.332 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 9, 2024-06-01T02:08:29.332 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 8 2024-06-01T02:08:29.332 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T02:08:29.332 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:08:29.680 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:08:29.681 INFO:teuthology.orchestra.run.smithi116.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-01T02:08:29.681 INFO:teuthology.orchestra.run.smithi116.stdout: "in_progress": true, 2024-06-01T02:08:29.681 INFO:teuthology.orchestra.run.smithi116.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-01T02:08:29.681 INFO:teuthology.orchestra.run.smithi116.stdout: "services_complete": [ 2024-06-01T02:08:29.681 INFO:teuthology.orchestra.run.smithi116.stdout: "mon", 2024-06-01T02:08:29.681 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr" 2024-06-01T02:08:29.681 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-06-01T02:08:29.681 INFO:teuthology.orchestra.run.smithi116.stdout: "progress": "8/23 daemons upgraded", 2024-06-01T02:08:29.681 INFO:teuthology.orchestra.run.smithi116.stdout: "message": "Currently upgrading osd daemons", 2024-06-01T02:08:29.681 INFO:teuthology.orchestra.run.smithi116.stdout: "is_paused": false 2024-06-01T02:08:29.681 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:08:30.076 INFO:teuthology.orchestra.run.smithi116.stdout:HEALTH_OK 2024-06-01T02:08:30.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:29 smithi161 ceph-mon[84406]: from='client.64306 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:08:30.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:29 smithi161 ceph-mon[84406]: from='client.44346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:08:30.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:29 smithi161 ceph-mon[84406]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-06-01T02:08:30.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:29 smithi161 ceph-mon[84406]: Upgrade: osd.3 is safe to restart 2024-06-01T02:08:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:29 smithi161 ceph-mon[84406]: from='client.64318 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:08:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:29 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/2498169136' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:30.242 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:29 smithi116 ceph-mon[100606]: from='client.64306 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:08:30.242 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:29 smithi116 ceph-mon[100606]: from='client.44346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:08:30.243 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:29 smithi116 ceph-mon[100606]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-06-01T02:08:30.243 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:29 smithi116 ceph-mon[100606]: Upgrade: osd.3 is safe to restart 2024-06-01T02:08:30.243 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:29 smithi116 ceph-mon[100606]: from='client.64318 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:08:30.243 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:29 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/2498169136' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:30.243 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:29 smithi116 ceph-mon[89502]: from='client.64306 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:08:30.244 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:29 smithi116 ceph-mon[89502]: from='client.44346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:08:30.244 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:29 smithi116 ceph-mon[89502]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-06-01T02:08:30.244 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:29 smithi116 ceph-mon[89502]: Upgrade: osd.3 is safe to restart 2024-06-01T02:08:30.244 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:29 smithi116 ceph-mon[89502]: from='client.64318 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:08:30.244 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:29 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/2498169136' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:31.158 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:30 smithi116 ceph-mon[89502]: pgmap v155: 161 pgs: 161 active+clean; 585 KiB data, 184 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 0 B/s, 3 objects/s recovering 2024-06-01T02:08:31.158 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:30 smithi116 ceph-mon[89502]: from='client.64330 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:08:31.158 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:30 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/505999734' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:08:31.158 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:31.158 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-01T02:08:31.158 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:31.158 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:31.158 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:08:31.158 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:30 smithi116 ceph-mon[100606]: pgmap v155: 161 pgs: 161 active+clean; 585 KiB data, 184 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 0 B/s, 3 objects/s recovering 2024-06-01T02:08:31.159 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:30 smithi116 ceph-mon[100606]: from='client.64330 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:08:31.159 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:30 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/505999734' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:08:31.159 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:31.159 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-01T02:08:31.159 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:31.159 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:31.159 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:08:31.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:30 smithi161 ceph-mon[84406]: pgmap v155: 161 pgs: 161 active+clean; 585 KiB data, 184 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 0 B/s, 3 objects/s recovering 2024-06-01T02:08:31.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:30 smithi161 ceph-mon[84406]: from='client.64330 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:08:31.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:30 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/505999734' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:08:31.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:31.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-01T02:08:31.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:31.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:31.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:08:32.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:31 smithi161 ceph-mon[84406]: Upgrade: Updating osd.3 2024-06-01T02:08:32.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:31 smithi161 ceph-mon[84406]: Deploying daemon osd.3 on smithi116 2024-06-01T02:08:32.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:32.266 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:31 smithi116 ceph-mon[100606]: Upgrade: Updating osd.3 2024-06-01T02:08:32.266 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:31 smithi116 ceph-mon[100606]: Deploying daemon osd.3 on smithi116 2024-06-01T02:08:32.266 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:32.267 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:31 smithi116 ceph-mon[89502]: Upgrade: Updating osd.3 2024-06-01T02:08:32.267 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:31 smithi116 ceph-mon[89502]: Deploying daemon osd.3 on smithi116 2024-06-01T02:08:32.267 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:33.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:32 smithi161 ceph-mon[84406]: pgmap v156: 161 pgs: 161 active+clean; 585 KiB data, 184 MiB used, 715 GiB / 715 GiB avail; 904 B/s rd, 0 op/s; 0 B/s, 3 objects/s recovering 2024-06-01T02:08:33.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:32 smithi116 ceph-mon[89502]: pgmap v156: 161 pgs: 161 active+clean; 585 KiB data, 184 MiB used, 715 GiB / 715 GiB avail; 904 B/s rd, 0 op/s; 0 B/s, 3 objects/s recovering 2024-06-01T02:08:33.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:32 smithi116 ceph-mon[100606]: pgmap v156: 161 pgs: 161 active+clean; 585 KiB data, 184 MiB used, 715 GiB / 715 GiB avail; 904 B/s rd, 0 op/s; 0 B/s, 3 objects/s recovering 2024-06-01T02:08:34.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:33 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:08:34.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:33 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:08:34.044 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:33 smithi116 systemd[1]: Stopping Ceph osd.3 for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:08:34.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:33 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:08:35.123 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:34 smithi116 ceph-mon[89502]: pgmap v157: 161 pgs: 161 active+clean; 585 KiB data, 184 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 3 objects/s recovering 2024-06-01T02:08:35.124 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:34 smithi116 ceph-mon[100606]: pgmap v157: 161 pgs: 161 active+clean; 585 KiB data, 184 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 3 objects/s recovering 2024-06-01T02:08:35.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:34 smithi161 ceph-mon[84406]: pgmap v157: 161 pgs: 161 active+clean; 585 KiB data, 184 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 3 objects/s recovering 2024-06-01T02:08:35.543 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:35 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3[42191]: 2024-06-01T02:08:35.127+0000 7f0d01b30700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-01T02:08:35.543 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:35 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3[42191]: 2024-06-01T02:08:35.127+0000 7f0d01b30700 -1 osd.3 143 *** Got signal Terminated *** 2024-06-01T02:08:35.543 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:35 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3[42191]: 2024-06-01T02:08:35.128+0000 7f0d01b30700 -1 osd.3 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-01T02:08:36.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:35 smithi116 ceph-mon[89502]: osd.3 marked itself down and dead 2024-06-01T02:08:36.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:35 smithi116 ceph-mon[100606]: osd.3 marked itself down and dead 2024-06-01T02:08:36.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:35 smithi161 ceph-mon[84406]: osd.3 marked itself down and dead 2024-06-01T02:08:36.749 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:36 smithi116 podman[126026]: 2024-06-01 02:08:36.477708197 +0000 UTC m=+1.466638808 container died a802290f52901f52e1238be6b1dc0a8a098b8e413b7a160db45ddc179c655975 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3, io.openshift.tags=base centos centos-stream, architecture=x86_64, vcs-type=git, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, io.k8s.display-name=CentOS Stream 8, CEPH_POINT_RELEASE=-17.2.0, com.redhat.component=centos-stream-container, version=8, GIT_REPO=https://github.com/ceph/ceph-container.git, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, release=754, vendor=Red Hat, Inc., com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., RELEASE=HEAD, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, io.buildah.version=1.19.8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, distribution-scope=public, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, name=centos-stream, build-date=2022-05-03T08:36:31.336870, ceph=True, io.openshift.expose-services=) 2024-06-01T02:08:37.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:36 smithi116 ceph-mon[89502]: pgmap v158: 161 pgs: 161 active+clean; 585 KiB data, 184 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:08:37.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:36 smithi116 ceph-mon[89502]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:08:37.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:36 smithi116 ceph-mon[89502]: osdmap e144: 8 total, 7 up, 8 in 2024-06-01T02:08:37.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:36 smithi116 ceph-mon[100606]: pgmap v158: 161 pgs: 161 active+clean; 585 KiB data, 184 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:08:37.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:36 smithi116 ceph-mon[100606]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:08:37.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:36 smithi116 ceph-mon[100606]: osdmap e144: 8 total, 7 up, 8 in 2024-06-01T02:08:37.044 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:36 smithi116 podman[126026]: 2024-06-01 02:08:36.979217913 +0000 UTC m=+1.968148507 container cleanup a802290f52901f52e1238be6b1dc0a8a098b8e413b7a160db45ddc179c655975 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3, com.redhat.component=centos-stream-container, distribution-scope=public, name=centos-stream, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, CEPH_POINT_RELEASE=-17.2.0, io.buildah.version=1.19.8, release=754, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-type=git, io.k8s.display-name=CentOS Stream 8, io.openshift.tags=base centos centos-stream, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, GIT_CLEAN=True, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, maintainer=Guillaume Abrioux , version=8, build-date=2022-05-03T08:36:31.336870, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, architecture=x86_64, ceph=True, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, vendor=Red Hat, Inc., GIT_BRANCH=HEAD, io.openshift.expose-services=) 2024-06-01T02:08:37.044 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:36 smithi116 bash[126026]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3 2024-06-01T02:08:37.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:08:36 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:08:36] "GET /metrics HTTP/1.1" 200 38443 "" "Prometheus/2.33.4" 2024-06-01T02:08:37.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:36 smithi161 ceph-mon[84406]: pgmap v158: 161 pgs: 161 active+clean; 585 KiB data, 184 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:08:37.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:36 smithi161 ceph-mon[84406]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:08:37.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:36 smithi161 ceph-mon[84406]: osdmap e144: 8 total, 7 up, 8 in 2024-06-01T02:08:37.670 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:37 smithi116 podman[126099]: 2024-06-01 02:08:37.33963399 +0000 UTC m=+0.852185962 container remove a802290f52901f52e1238be6b1dc0a8a098b8e413b7a160db45ddc179c655975 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3, ceph=True, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, architecture=x86_64, io.openshift.tags=base centos centos-stream, maintainer=Guillaume Abrioux , name=centos-stream, vendor=Red Hat, Inc., io.buildah.version=1.19.8, release=754, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, distribution-scope=public, GIT_REPO=https://github.com/ceph/ceph-container.git, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, GIT_CLEAN=True, io.k8s.display-name=CentOS Stream 8, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, GIT_BRANCH=HEAD, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-type=git, build-date=2022-05-03T08:36:31.336870, version=8, RELEASE=HEAD, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, com.redhat.component=centos-stream-container, io.openshift.expose-services=, CEPH_POINT_RELEASE=-17.2.0) 2024-06-01T02:08:38.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:37 smithi116 ceph-mon[89502]: osdmap e145: 8 total, 7 up, 8 in 2024-06-01T02:08:38.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:37 smithi116 ceph-mon[89502]: pgmap v161: 161 pgs: 11 active+undersized+degraded, 39 peering, 18 active+undersized, 93 active+clean; 585 KiB data, 184 MiB used, 715 GiB / 715 GiB avail; 50/726 objects degraded (6.887%) 2024-06-01T02:08:38.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:37 smithi116 ceph-mon[100606]: osdmap e145: 8 total, 7 up, 8 in 2024-06-01T02:08:38.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:37 smithi116 ceph-mon[100606]: pgmap v161: 161 pgs: 11 active+undersized+degraded, 39 peering, 18 active+undersized, 93 active+clean; 585 KiB data, 184 MiB used, 715 GiB / 715 GiB avail; 50/726 objects degraded (6.887%) 2024-06-01T02:08:38.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:37 smithi161 ceph-mon[84406]: osdmap e145: 8 total, 7 up, 8 in 2024-06-01T02:08:38.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:37 smithi161 ceph-mon[84406]: pgmap v161: 161 pgs: 11 active+undersized+degraded, 39 peering, 18 active+undersized, 93 active+clean; 585 KiB data, 184 MiB used, 715 GiB / 715 GiB avail; 50/726 objects degraded (6.887%) 2024-06-01T02:08:38.575 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:38 smithi116 podman[126328]: 2024-06-01 02:08:38.227450592 +0000 UTC m=+0.018592200 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:08:38.575 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:38 smithi116 podman[126328]: 2024-06-01 02:08:38.344255903 +0000 UTC m=+0.135397477 container create e95bd9641dbaf53dd9418f950fa1b92f4f3547a72ed7858d4889f1a1c23c378c (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-06-01T02:08:38.864 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:38 smithi116 podman[126328]: 2024-06-01 02:08:38.580017862 +0000 UTC m=+0.371159445 container init e95bd9641dbaf53dd9418f950fa1b92f4f3547a72ed7858d4889f1a1c23c378c (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-06-01T02:08:38.865 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:38 smithi116 podman[126328]: 2024-06-01 02:08:38.583497869 +0000 UTC m=+0.374639455 container start e95bd9641dbaf53dd9418f950fa1b92f4f3547a72ed7858d4889f1a1c23c378c (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True) 2024-06-01T02:08:38.865 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:38 smithi116 podman[126328]: 2024-06-01 02:08:38.663633518 +0000 UTC m=+0.454775113 container attach e95bd9641dbaf53dd9418f950fa1b92f4f3547a72ed7858d4889f1a1c23c378c (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-06-01T02:08:39.255 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:38 smithi116 ceph-mon[100606]: Health check failed: Reduced data availability: 5 pgs inactive, 7 pgs peering (PG_AVAILABILITY) 2024-06-01T02:08:39.255 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:38 smithi116 ceph-mon[100606]: Health check failed: Degraded data redundancy: 50/726 objects degraded (6.887%), 11 pgs degraded (PG_DEGRADED) 2024-06-01T02:08:39.255 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:38 smithi116 ceph-mon[89502]: Health check failed: Reduced data availability: 5 pgs inactive, 7 pgs peering (PG_AVAILABILITY) 2024-06-01T02:08:39.255 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:38 smithi116 ceph-mon[89502]: Health check failed: Degraded data redundancy: 50/726 objects degraded (6.887%), 11 pgs degraded (PG_DEGRADED) 2024-06-01T02:08:39.255 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:38 smithi116 podman[126328]: 2024-06-01 02:08:38.900574101 +0000 UTC m=+0.691715692 container died e95bd9641dbaf53dd9418f950fa1b92f4f3547a72ed7858d4889f1a1c23c378c (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-06-01T02:08:39.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:38 smithi161 ceph-mon[84406]: Health check failed: Reduced data availability: 5 pgs inactive, 7 pgs peering (PG_AVAILABILITY) 2024-06-01T02:08:39.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:38 smithi161 ceph-mon[84406]: Health check failed: Degraded data redundancy: 50/726 objects degraded (6.887%), 11 pgs degraded (PG_DEGRADED) 2024-06-01T02:08:39.963 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:39 smithi116 podman[126408]: 2024-06-01 02:08:39.651632658 +0000 UTC m=+0.746094243 container remove e95bd9641dbaf53dd9418f950fa1b92f4f3547a72ed7858d4889f1a1c23c378c (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-01T02:08:39.964 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:39 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.3.service: Deactivated successfully. 2024-06-01T02:08:39.964 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:39 smithi116 systemd[1]: Stopped Ceph osd.3 for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:08:39.964 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:39 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.3.service: Consumed 11.066s CPU time. 2024-06-01T02:08:39.964 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:39 smithi116 systemd[1]: Starting Ceph osd.3 for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:08:40.177 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:39 smithi161 ceph-mon[84406]: pgmap v162: 161 pgs: 11 active+undersized+degraded, 39 peering, 18 active+undersized, 93 active+clean; 585 KiB data, 184 MiB used, 715 GiB / 715 GiB avail; 50/726 objects degraded (6.887%) 2024-06-01T02:08:40.223 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:39 smithi116 ceph-mon[89502]: pgmap v162: 161 pgs: 11 active+undersized+degraded, 39 peering, 18 active+undersized, 93 active+clean; 585 KiB data, 184 MiB used, 715 GiB / 715 GiB avail; 50/726 objects degraded (6.887%) 2024-06-01T02:08:40.223 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:39 smithi116 ceph-mon[100606]: pgmap v162: 161 pgs: 11 active+undersized+degraded, 39 peering, 18 active+undersized, 93 active+clean; 585 KiB data, 184 MiB used, 715 GiB / 715 GiB avail; 50/726 objects degraded (6.887%) 2024-06-01T02:08:40.224 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:40 smithi116 podman[126577]: 2024-06-01 02:08:40.013444203 +0000 UTC m=+0.017762899 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:08:40.484 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:40 smithi116 podman[126577]: 2024-06-01 02:08:40.228382362 +0000 UTC m=+0.232701030 container create 09c4e576954811f60023f93170220e55d3786ee5bd2c3547fe3a1ca136451763 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3-activate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-06-01T02:08:40.793 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:40 smithi116 podman[126577]: 2024-06-01 02:08:40.48967871 +0000 UTC m=+0.493997392 container init 09c4e576954811f60023f93170220e55d3786ee5bd2c3547fe3a1ca136451763 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3-activate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-06-01T02:08:40.793 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:40 smithi116 podman[126577]: 2024-06-01 02:08:40.493043275 +0000 UTC m=+0.497361947 container start 09c4e576954811f60023f93170220e55d3786ee5bd2c3547fe3a1ca136451763 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3-activate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-01T02:08:40.793 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:40 smithi116 podman[126577]: 2024-06-01 02:08:40.572985018 +0000 UTC m=+0.577303692 container attach 09c4e576954811f60023f93170220e55d3786ee5bd2c3547fe3a1ca136451763 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3-activate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc) 2024-06-01T02:08:41.223 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:40 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3-activate[126604]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-06-01T02:08:41.223 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:40 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3-activate[126604]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_1 2024-06-01T02:08:41.223 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:40 smithi116 bash[126577]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-06-01T02:08:41.223 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:40 smithi116 bash[126577]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_1 2024-06-01T02:08:41.223 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:40 smithi116 bash[126577]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_1 2024-06-01T02:08:41.223 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:40 smithi116 bash[126577]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0 2024-06-01T02:08:41.223 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:40 smithi116 bash[126577]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-3/block 2024-06-01T02:08:41.223 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:40 smithi116 bash[126577]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-06-01T02:08:41.223 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:40 smithi116 bash[126577]: --> ceph-volume raw activate successful for osd ID: 3 2024-06-01T02:08:41.223 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:40 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3-activate[126604]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_1 2024-06-01T02:08:41.224 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:40 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3-activate[126604]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0 2024-06-01T02:08:41.224 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:40 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3-activate[126604]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-3/block 2024-06-01T02:08:41.224 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:40 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3-activate[126604]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-06-01T02:08:41.224 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:40 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3-activate[126604]: --> ceph-volume raw activate successful for osd ID: 3 2024-06-01T02:08:41.224 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:40 smithi116 podman[126577]: 2024-06-01 02:08:40.993160022 +0000 UTC m=+0.997478695 container died 09c4e576954811f60023f93170220e55d3786ee5bd2c3547fe3a1ca136451763 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, io.buildah.version=1.33.5) 2024-06-01T02:08:41.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:41.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:41.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:41.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:41.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:41.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:41.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:41.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:41.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:42.292 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:42 smithi116 podman[126809]: 2024-06-01 02:08:42.073215394 +0000 UTC m=+1.092234877 container remove 09c4e576954811f60023f93170220e55d3786ee5bd2c3547fe3a1ca136451763 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3-activate, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-01T02:08:42.707 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:42 smithi116 ceph-mon[89502]: pgmap v163: 161 pgs: 11 active+undersized+degraded, 39 peering, 18 active+undersized, 93 active+clean; 585 KiB data, 184 MiB used, 715 GiB / 715 GiB avail; 50/726 objects degraded (6.887%) 2024-06-01T02:08:42.708 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:42.708 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:42.708 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:42.708 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:42 smithi116 ceph-mon[100606]: pgmap v163: 161 pgs: 11 active+undersized+degraded, 39 peering, 18 active+undersized, 93 active+clean; 585 KiB data, 184 MiB used, 715 GiB / 715 GiB avail; 50/726 objects degraded (6.887%) 2024-06-01T02:08:42.708 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:42.708 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:42.708 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:42.708 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:42 smithi116 podman[126963]: 2024-06-01 02:08:42.542666863 +0000 UTC m=+0.111253427 container create bd00ad2c25566eeec3d5acf3c02e7d6ebf64fc5dd23a587cf8f36bce13deeb8f (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-06-01T02:08:42.709 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:42 smithi116 podman[126963]: 2024-06-01 02:08:42.449067943 +0000 UTC m=+0.017654536 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:08:42.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:42 smithi161 ceph-mon[84406]: pgmap v163: 161 pgs: 11 active+undersized+degraded, 39 peering, 18 active+undersized, 93 active+clean; 585 KiB data, 184 MiB used, 715 GiB / 715 GiB avail; 50/726 objects degraded (6.887%) 2024-06-01T02:08:42.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:42.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:42.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:43.454 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:43 smithi116 podman[126963]: 2024-06-01 02:08:43.180957401 +0000 UTC m=+0.749543987 container init bd00ad2c25566eeec3d5acf3c02e7d6ebf64fc5dd23a587cf8f36bce13deeb8f (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc) 2024-06-01T02:08:43.454 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:43 smithi116 podman[126963]: 2024-06-01 02:08:43.184470407 +0000 UTC m=+0.753056987 container start bd00ad2c25566eeec3d5acf3c02e7d6ebf64fc5dd23a587cf8f36bce13deeb8f (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-06-01T02:08:43.454 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:43 smithi116 ceph-osd[127029]: -- 172.21.15.116:0/2957545196 <== mon.0 v2:172.21.15.116:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c02fc1d680 con 0x55c02fbe6c00 2024-06-01T02:08:43.454 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:43 smithi116 bash[126963]: bd00ad2c25566eeec3d5acf3c02e7d6ebf64fc5dd23a587cf8f36bce13deeb8f 2024-06-01T02:08:43.454 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:43 smithi116 systemd[1]: Started Ceph osd.3 for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:08:43.722 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:43 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:08:43.722 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:43.722 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:43.723 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:43.723 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:43.723 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:43.723 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:43.723 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:43.723 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:43.723 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:43.723 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:43.723 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:43 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:08:43.723 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:43.723 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:43.723 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:43.724 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:43.724 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:43.724 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:43.724 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:43.724 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:43.724 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:43.724 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:43.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:43 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:08:43.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:43.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:43.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:43.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:43.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:43.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:43.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:43.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:43.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:43.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:43.991 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:43 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3[126981]: 2024-06-01T02:08:43.725+0000 7f4c591697c0 -1 Falling back to public interface 2024-06-01T02:08:44.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:44.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: pgmap v164: 161 pgs: 25 active+undersized+degraded, 43 active+undersized, 93 active+clean; 585 KiB data, 185 MiB used, 715 GiB / 715 GiB avail; 93/726 objects degraded (12.810%) 2024-06-01T02:08:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:44.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:44.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:44.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:44.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:44.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:44.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:44.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:44.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:44.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:44.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:44.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:44.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:44.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:44.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:44.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:44.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:44.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:44.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:44.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:44.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:44.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:44.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: Health check update: Degraded data redundancy: 93/726 objects degraded (12.810%), 25 pgs degraded (PG_DEGRADED) 2024-06-01T02:08:44.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs inactive, 7 pgs peering) 2024-06-01T02:08:44.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:44.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:44.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:44.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:44.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:44.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:44.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:44.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:44.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:44.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:44.960 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:44.960 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: pgmap v164: 161 pgs: 25 active+undersized+degraded, 43 active+undersized, 93 active+clean; 585 KiB data, 185 MiB used, 715 GiB / 715 GiB avail; 93/726 objects degraded (12.810%) 2024-06-01T02:08:44.960 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:44.960 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:44.960 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:44.960 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:44.960 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:44.960 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:44.960 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:44.960 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:44.960 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:44.961 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:44.961 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:44.961 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:44.961 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:44.961 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:44.961 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:44.961 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:44.961 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:44.961 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:44.961 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:44.961 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:44.961 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:44.961 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:44.961 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:44.962 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:44.962 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:44.962 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:44.962 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:44.962 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: pgmap v164: 161 pgs: 25 active+undersized+degraded, 43 active+undersized, 93 active+clean; 585 KiB data, 185 MiB used, 715 GiB / 715 GiB avail; 93/726 objects degraded (12.810%) 2024-06-01T02:08:44.962 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:44.962 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:44.962 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:44.962 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:44.963 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:44.963 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:44.963 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:44.963 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:44.963 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:44.963 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:44.963 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:44.963 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:44.963 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:44.963 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:44.963 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:44.963 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:44.963 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:44.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:44.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:44.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:44.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:44.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:44.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:44.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:44.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:44.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:44.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:44.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:44.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:44.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:44.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:44.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:44.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:44.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:44.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:44.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:44.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:44.965 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:44.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:44.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:44.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:44.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:44.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:44.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:44.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:44.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:44.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:44.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: Health check update: Degraded data redundancy: 93/726 objects degraded (12.810%), 25 pgs degraded (PG_DEGRADED) 2024-06-01T02:08:44.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs inactive, 7 pgs peering) 2024-06-01T02:08:44.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:44.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:44.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:44.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:44.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:44.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:44.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:44.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:44.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:44.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:44.967 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:44.967 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:44.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:44.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:44.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:44.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:44.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:44.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:44.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:44.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:44.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:44.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:44.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:44.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:44.968 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:44.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:44.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:44.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:44.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:44.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: Health check update: Degraded data redundancy: 93/726 objects degraded (12.810%), 25 pgs degraded (PG_DEGRADED) 2024-06-01T02:08:44.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs inactive, 7 pgs peering) 2024-06-01T02:08:44.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:44.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:44.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:44.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:44.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:44.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:44.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:44.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:44.970 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:44.970 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:45.558 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:08:45.298+0000 7fcedaabb700 -1 mgr.server reply reply (16) Device or resource busy unsafe to stop osd(s) at this time (20 PGs are or would become offline) 2024-06-01T02:08:45.878 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:45.879 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:45.879 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:45.879 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:45.879 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:45.879 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:45.879 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:45.879 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:45.879 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:45.879 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:45.879 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:45.879 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:45.879 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:45.879 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:45.880 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:45.880 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:45.880 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:45.880 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:45.880 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:45.880 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:45.880 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:45.880 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:45.880 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:45.880 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:45.880 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:45.880 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:45.881 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:45.882 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:45.882 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:45.882 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:45.882 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:45.882 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:45.882 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:45.882 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:45.883 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:45.883 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:45.883 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:45.883 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:45.883 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:45.883 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:45.883 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:45.883 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:45.883 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:45.883 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:45.883 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:45.883 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:45.883 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:45.883 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:45.884 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:45.884 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:45.884 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:45.884 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:45.884 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:45.884 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:45.884 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:45.884 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:45.884 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:45.884 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:45.884 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:45.885 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:45.885 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:45.885 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:45.885 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:45.885 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:45.885 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:45.885 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:45.885 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:45.885 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:45.885 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:45.886 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:45.886 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:45.886 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:45.886 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:45.886 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:45.886 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:45.886 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-01T02:08:45.886 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:45.886 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:45.886 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:45.886 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:45.887 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:45.887 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:45.887 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:45.887 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:45.887 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:45.887 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:45.887 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:45.887 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:45.887 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:45.887 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:45.887 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:45.887 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-01T02:08:45.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:45.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:45.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:45.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:45.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:45.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:45.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:45.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:45.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:45.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:45.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:45.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:45.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:45.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:45.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:45.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:45.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:08:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:08:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:08:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:45.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:08:45.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-01T02:08:46.728 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:46 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3[126981]: 2024-06-01T02:08:46.479+0000 7f4c591697c0 -1 osd.3 0 read_superblock omap replica is missing. 2024-06-01T02:08:46.728 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:46 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3[126981]: 2024-06-01T02:08:46.513+0000 7f4c591697c0 -1 osd.3 143 log_to_monitors true 2024-06-01T02:08:47.042 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:47.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:47.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:47.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:46 smithi116 ceph-mon[100606]: pgmap v165: 161 pgs: 25 active+undersized+degraded, 43 active+undersized, 93 active+clean; 585 KiB data, 185 MiB used, 715 GiB / 715 GiB avail; 93/726 objects degraded (12.810%) 2024-06-01T02:08:47.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:46 smithi116 ceph-mon[100606]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-01T02:08:47.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:46 smithi116 ceph-mon[100606]: Upgrade: unsafe to stop osd(s) at this time (20 PGs are or would become offline) 2024-06-01T02:08:47.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:47.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:08:47.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:46 smithi116 ceph-mon[100606]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-06-01T02:08:47.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:46 smithi116 ceph-mon[100606]: from='osd.3 [v2:172.21.15.116:6824/373446283,v1:172.21.15.116:6825/373446283]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-01T02:08:47.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:47.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:47.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:47.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:46 smithi116 ceph-mon[89502]: pgmap v165: 161 pgs: 25 active+undersized+degraded, 43 active+undersized, 93 active+clean; 585 KiB data, 185 MiB used, 715 GiB / 715 GiB avail; 93/726 objects degraded (12.810%) 2024-06-01T02:08:47.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:46 smithi116 ceph-mon[89502]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-01T02:08:47.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:46 smithi116 ceph-mon[89502]: Upgrade: unsafe to stop osd(s) at this time (20 PGs are or would become offline) 2024-06-01T02:08:47.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:47.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:08:47.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:46 smithi116 ceph-mon[89502]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-06-01T02:08:47.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:46 smithi116 ceph-mon[89502]: from='osd.3 [v2:172.21.15.116:6824/373446283,v1:172.21.15.116:6825/373446283]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-01T02:08:47.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:08:46 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:08:46] "GET /metrics HTTP/1.1" 200 38447 "" "Prometheus/2.33.4" 2024-06-01T02:08:47.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:46 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:47.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:46 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:47.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:46 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:08:47.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:46 smithi161 ceph-mon[84406]: pgmap v165: 161 pgs: 25 active+undersized+degraded, 43 active+undersized, 93 active+clean; 585 KiB data, 185 MiB used, 715 GiB / 715 GiB avail; 93/726 objects degraded (12.810%) 2024-06-01T02:08:47.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:46 smithi161 ceph-mon[84406]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-01T02:08:47.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:46 smithi161 ceph-mon[84406]: Upgrade: unsafe to stop osd(s) at this time (20 PGs are or would become offline) 2024-06-01T02:08:47.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:47.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:08:47.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:46 smithi161 ceph-mon[84406]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-06-01T02:08:47.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:46 smithi161 ceph-mon[84406]: from='osd.3 [v2:172.21.15.116:6824/373446283,v1:172.21.15.116:6825/373446283]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-01T02:08:47.363 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:08:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3[126981]: 2024-06-01T02:08:47.343+0000 7f4c4a5c0700 -1 osd.3 143 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-01T02:08:48.646 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:48 smithi116 ceph-mon[100606]: pgmap v166: 161 pgs: 25 active+undersized+degraded, 43 active+undersized, 93 active+clean; 585 KiB data, 185 MiB used, 715 GiB / 715 GiB avail; 695 B/s rd, 0 op/s; 93/726 objects degraded (12.810%) 2024-06-01T02:08:48.647 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:48 smithi116 ceph-mon[100606]: from='osd.3 [v2:172.21.15.116:6824/373446283,v1:172.21.15.116:6825/373446283]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-01T02:08:48.647 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:48 smithi116 ceph-mon[100606]: osdmap e146: 8 total, 7 up, 8 in 2024-06-01T02:08:48.647 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:48 smithi116 ceph-mon[100606]: from='osd.3 [v2:172.21.15.116:6824/373446283,v1:172.21.15.116:6825/373446283]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T02:08:48.647 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:48 smithi116 ceph-mon[89502]: pgmap v166: 161 pgs: 25 active+undersized+degraded, 43 active+undersized, 93 active+clean; 585 KiB data, 185 MiB used, 715 GiB / 715 GiB avail; 695 B/s rd, 0 op/s; 93/726 objects degraded (12.810%) 2024-06-01T02:08:48.647 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:48 smithi116 ceph-mon[89502]: from='osd.3 [v2:172.21.15.116:6824/373446283,v1:172.21.15.116:6825/373446283]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-01T02:08:48.647 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:48 smithi116 ceph-mon[89502]: osdmap e146: 8 total, 7 up, 8 in 2024-06-01T02:08:48.647 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:48 smithi116 ceph-mon[89502]: from='osd.3 [v2:172.21.15.116:6824/373446283,v1:172.21.15.116:6825/373446283]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T02:08:48.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:48 smithi161 ceph-mon[84406]: pgmap v166: 161 pgs: 25 active+undersized+degraded, 43 active+undersized, 93 active+clean; 585 KiB data, 185 MiB used, 715 GiB / 715 GiB avail; 695 B/s rd, 0 op/s; 93/726 objects degraded (12.810%) 2024-06-01T02:08:48.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:48 smithi161 ceph-mon[84406]: from='osd.3 [v2:172.21.15.116:6824/373446283,v1:172.21.15.116:6825/373446283]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-01T02:08:48.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:48 smithi161 ceph-mon[84406]: osdmap e146: 8 total, 7 up, 8 in 2024-06-01T02:08:48.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:48 smithi161 ceph-mon[84406]: from='osd.3 [v2:172.21.15.116:6824/373446283,v1:172.21.15.116:6825/373446283]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-06-01T02:08:49.612 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:49 smithi116 ceph-mon[89502]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:08:49.612 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:49 smithi116 ceph-mon[89502]: osd.3 [v2:172.21.15.116:6824/373446283,v1:172.21.15.116:6825/373446283] boot 2024-06-01T02:08:49.612 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:49 smithi116 ceph-mon[89502]: osdmap e147: 8 total, 8 up, 8 in 2024-06-01T02:08:49.612 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T02:08:49.612 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:49 smithi116 ceph-mon[89502]: osdmap e148: 8 total, 8 up, 8 in 2024-06-01T02:08:49.613 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:49 smithi116 ceph-mon[100606]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:08:49.613 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:49 smithi116 ceph-mon[100606]: osd.3 [v2:172.21.15.116:6824/373446283,v1:172.21.15.116:6825/373446283] boot 2024-06-01T02:08:49.613 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:49 smithi116 ceph-mon[100606]: osdmap e147: 8 total, 8 up, 8 in 2024-06-01T02:08:49.613 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T02:08:49.613 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:49 smithi116 ceph-mon[100606]: osdmap e148: 8 total, 8 up, 8 in 2024-06-01T02:08:49.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:49 smithi161 ceph-mon[84406]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:08:49.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:49 smithi161 ceph-mon[84406]: osd.3 [v2:172.21.15.116:6824/373446283,v1:172.21.15.116:6825/373446283] boot 2024-06-01T02:08:49.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:49 smithi161 ceph-mon[84406]: osdmap e147: 8 total, 8 up, 8 in 2024-06-01T02:08:49.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-01T02:08:49.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:49 smithi161 ceph-mon[84406]: osdmap e148: 8 total, 8 up, 8 in 2024-06-01T02:08:50.542 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:50 smithi116 ceph-mon[89502]: pgmap v169: 161 pgs: 25 active+undersized+degraded, 43 active+undersized, 93 active+clean; 585 KiB data, 195 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 93/726 objects degraded (12.810%) 2024-06-01T02:08:50.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:50 smithi116 ceph-mon[100606]: pgmap v169: 161 pgs: 25 active+undersized+degraded, 43 active+undersized, 93 active+clean; 585 KiB data, 195 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 93/726 objects degraded (12.810%) 2024-06-01T02:08:50.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:50 smithi161 ceph-mon[84406]: pgmap v169: 161 pgs: 25 active+undersized+degraded, 43 active+undersized, 93 active+clean; 585 KiB data, 195 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 93/726 objects degraded (12.810%) 2024-06-01T02:08:52.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:52 smithi161 ceph-mon[84406]: pgmap v171: 161 pgs: 25 active+undersized+degraded, 43 active+undersized, 93 active+clean; 585 KiB data, 195 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 93/726 objects degraded (12.810%) 2024-06-01T02:08:52.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:52.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:52.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:52.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:52 smithi116 ceph-mon[100606]: pgmap v171: 161 pgs: 25 active+undersized+degraded, 43 active+undersized, 93 active+clean; 585 KiB data, 195 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 93/726 objects degraded (12.810%) 2024-06-01T02:08:52.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:52.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:52.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:52.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:52 smithi116 ceph-mon[89502]: pgmap v171: 161 pgs: 25 active+undersized+degraded, 43 active+undersized, 93 active+clean; 585 KiB data, 195 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 93/726 objects degraded (12.810%) 2024-06-01T02:08:52.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:52.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:52.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:53.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:53 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:08:53.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:53 smithi161 ceph-mon[84406]: pgmap v172: 161 pgs: 17 active+undersized+degraded, 27 active+undersized, 117 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 71/726 objects degraded (9.780%) 2024-06-01T02:08:53.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:53.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:53.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:53.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:53 smithi161 ceph-mon[84406]: Health check update: Degraded data redundancy: 71/726 objects degraded (9.780%), 17 pgs degraded (PG_DEGRADED) 2024-06-01T02:08:53.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:53 smithi161 ceph-mon[84406]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-06-01T02:08:53.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:53 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:08:53.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:53 smithi116 ceph-mon[100606]: pgmap v172: 161 pgs: 17 active+undersized+degraded, 27 active+undersized, 117 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 71/726 objects degraded (9.780%) 2024-06-01T02:08:53.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:53.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:53.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:53.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:53 smithi116 ceph-mon[100606]: Health check update: Degraded data redundancy: 71/726 objects degraded (9.780%), 17 pgs degraded (PG_DEGRADED) 2024-06-01T02:08:53.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:53 smithi116 ceph-mon[100606]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-06-01T02:08:53.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:53 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:08:53.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:53 smithi116 ceph-mon[89502]: pgmap v172: 161 pgs: 17 active+undersized+degraded, 27 active+undersized, 117 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 71/726 objects degraded (9.780%) 2024-06-01T02:08:53.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:53.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:53.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:08:53.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:53 smithi116 ceph-mon[89502]: Health check update: Degraded data redundancy: 71/726 objects degraded (9.780%), 17 pgs degraded (PG_DEGRADED) 2024-06-01T02:08:53.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:53 smithi116 ceph-mon[89502]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-06-01T02:08:54.756 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:54 smithi161 ceph-mon[84406]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 71/726 objects degraded (9.780%), 17 pgs degraded) 2024-06-01T02:08:54.756 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:54 smithi161 ceph-mon[84406]: Cluster is now healthy 2024-06-01T02:08:54.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:54 smithi116 ceph-mon[89502]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 71/726 objects degraded (9.780%), 17 pgs degraded) 2024-06-01T02:08:54.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:54 smithi116 ceph-mon[89502]: Cluster is now healthy 2024-06-01T02:08:54.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:54 smithi116 ceph-mon[100606]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 71/726 objects degraded (9.780%), 17 pgs degraded) 2024-06-01T02:08:54.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:54 smithi116 ceph-mon[100606]: Cluster is now healthy 2024-06-01T02:08:55.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:55 smithi116 ceph-mon[89502]: pgmap v173: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:08:55.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:55 smithi116 ceph-mon[100606]: pgmap v173: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:08:55.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:55 smithi161 ceph-mon[84406]: pgmap v173: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:08:57.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:08:56 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:08:56] "GET /metrics HTTP/1.1" 200 38447 "" "Prometheus/2.33.4" 2024-06-01T02:08:57.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:57 smithi116 ceph-mon[89502]: pgmap v174: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:08:57.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:57 smithi116 ceph-mon[100606]: pgmap v174: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:08:57.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:57 smithi161 ceph-mon[84406]: pgmap v174: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:08:59.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:08:59 smithi116 ceph-mon[100606]: pgmap v175: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:08:59.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:08:59 smithi116 ceph-mon[89502]: pgmap v175: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:08:59.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:08:59 smithi161 ceph-mon[84406]: pgmap v175: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:09:00.441 INFO:teuthology.orchestra.run.smithi116.stdout:true 2024-06-01T02:09:00.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-01T02:09:00.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:00.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:00.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:00.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:00 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-01T02:09:00.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:00 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:00.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:00 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:00.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:00 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:00.901 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:00 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-01T02:09:00.902 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:00 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:00.902 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:00 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:00.902 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:00 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:01.095 INFO:teuthology.orchestra.run.smithi116.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-01T02:09:01.095 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi116 smithi116 running 8s ago 17m - - 2024-06-01T02:09:01.095 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi161 smithi161 running 0s ago 17m - - 2024-06-01T02:09:01.095 INFO:teuthology.orchestra.run.smithi116.stdout:alertmanager.a smithi116 *:9093,9094 running (7m) 8s ago 12m 21.2M - 0.23.0 ba2b418f427c e0c297162ded 2024-06-01T02:09:01.095 INFO:teuthology.orchestra.run.smithi116.stdout:grafana.a smithi161 *:3000 running (6m) 0s ago 11m 53.2M - 8.3.5 dad864ee21e9 00b0a78ee88e 2024-06-01T02:09:01.095 INFO:teuthology.orchestra.run.smithi116.stdout:iscsi.foo.smithi116.gjuxzb smithi116 running (5m) 8s ago 10m 56.7M - 3.5 e1d6a67b021e 0e191fe3e3bb 2024-06-01T02:09:01.095 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.x smithi161 *:8443,9283,8765 running (6m) 0s ago 16m 526M - 19.0.0-2946-g3d89accf b045ac545f82 2d7fbb85e5f7 2024-06-01T02:09:01.095 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.y smithi116 *:8443,9283,8765 running (5m) 8s ago 18m 454M - 19.0.0-2946-g3d89accf b045ac545f82 bc9627c27a57 2024-06-01T02:09:01.096 INFO:teuthology.orchestra.run.smithi116.stdout:mon.a smithi116 running (4m) 8s ago 18m 60.2M 2048M 19.0.0-2946-g3d89accf b045ac545f82 b5e9b6f873ff 2024-06-01T02:09:01.096 INFO:teuthology.orchestra.run.smithi116.stdout:mon.b smithi161 running (3m) 0s ago 17m 39.3M 2048M 19.0.0-2946-g3d89accf b045ac545f82 aaa08c03cfc8 2024-06-01T02:09:01.096 INFO:teuthology.orchestra.run.smithi116.stdout:mon.c smithi116 running (3m) 8s ago 17m 44.9M 2048M 19.0.0-2946-g3d89accf b045ac545f82 bd0fb0e603fa 2024-06-01T02:09:01.096 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.a smithi116 *:9100 running (7m) 8s ago 12m 17.7M - 1.3.1 1dbe0e931976 e347748d17bb 2024-06-01T02:09:01.096 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.b smithi161 *:9100 running (6m) 0s ago 12m 17.6M - 1.3.1 1dbe0e931976 6e66182f5a1a 2024-06-01T02:09:01.096 INFO:teuthology.orchestra.run.smithi116.stdout:osd.0 smithi116 running (118s) 8s ago 16m 47.1M 4096M 19.0.0-2946-g3d89accf b045ac545f82 b4313d121b9d 2024-06-01T02:09:01.096 INFO:teuthology.orchestra.run.smithi116.stdout:osd.1 smithi116 running (83s) 8s ago 16m 49.5M 4096M 19.0.0-2946-g3d89accf b045ac545f82 dccdbeac5aae 2024-06-01T02:09:01.096 INFO:teuthology.orchestra.run.smithi116.stdout:osd.2 smithi116 running (48s) 8s ago 15m 45.0M 4096M 19.0.0-2946-g3d89accf b045ac545f82 1ad95efda404 2024-06-01T02:09:01.096 INFO:teuthology.orchestra.run.smithi116.stdout:osd.3 smithi116 running (18s) 8s ago 15m 46.8M 4096M 19.0.0-2946-g3d89accf b045ac545f82 bd00ad2c2556 2024-06-01T02:09:01.096 INFO:teuthology.orchestra.run.smithi116.stdout:osd.4 smithi161 running (14m) 0s ago 14m 63.6M 4096M 17.2.0 e1d6a67b021e 70b3f89b0bd9 2024-06-01T02:09:01.096 INFO:teuthology.orchestra.run.smithi116.stdout:osd.5 smithi161 running (14m) 0s ago 14m 59.0M 4096M 17.2.0 e1d6a67b021e 68358b78a25d 2024-06-01T02:09:01.096 INFO:teuthology.orchestra.run.smithi116.stdout:osd.6 smithi161 running (14m) 0s ago 14m 58.9M 4096M 17.2.0 e1d6a67b021e 50abeb0551e7 2024-06-01T02:09:01.096 INFO:teuthology.orchestra.run.smithi116.stdout:osd.7 smithi161 running (13m) 0s ago 13m 62.5M 4096M 17.2.0 e1d6a67b021e 6004e6983735 2024-06-01T02:09:01.096 INFO:teuthology.orchestra.run.smithi116.stdout:prometheus.a smithi161 *:9095 running (5m) 0s ago 13m 76.9M - 2.33.4 514e6a882f6e e977f6ca1d73 2024-06-01T02:09:01.096 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi116.vzjrxj smithi116 *:8000 running (11m) 8s ago 11m 77.0M - 17.2.0 e1d6a67b021e df4fe3c1dbc3 2024-06-01T02:09:01.097 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi161.ashqxz smithi161 *:8000 running (11m) 0s ago 11m 77.7M - 17.2.0 e1d6a67b021e 3eecc1f3e2d3 2024-06-01T02:09:01.097 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi116.hokbmf smithi116 *:80 running (10m) 8s ago 10m 77.4M - 17.2.0 e1d6a67b021e 10950ed75dbd 2024-06-01T02:09:01.097 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi161.axjlzw smithi161 *:80 running (10m) 0s ago 10m 76.5M - 17.2.0 e1d6a67b021e 0b5d5ce47772 2024-06-01T02:09:01.489 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:09:01.489 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": { 2024-06-01T02:09:01.489 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 3 2024-06-01T02:09:01.489 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:09:01.489 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": { 2024-06-01T02:09:01.489 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-01T02:09:01.489 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:09:01.490 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": { 2024-06-01T02:09:01.490 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4, 2024-06-01T02:09:01.490 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 4 2024-06-01T02:09:01.490 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:09:01.490 INFO:teuthology.orchestra.run.smithi116.stdout: "rgw": { 2024-06-01T02:09:01.490 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-06-01T02:09:01.490 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:09:01.490 INFO:teuthology.orchestra.run.smithi116.stdout: "overall": { 2024-06-01T02:09:01.490 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8, 2024-06-01T02:09:01.490 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 9 2024-06-01T02:09:01.490 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T02:09:01.490 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:09:01.825 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:09:01.825 INFO:teuthology.orchestra.run.smithi116.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-01T02:09:01.825 INFO:teuthology.orchestra.run.smithi116.stdout: "in_progress": true, 2024-06-01T02:09:01.825 INFO:teuthology.orchestra.run.smithi116.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-01T02:09:01.825 INFO:teuthology.orchestra.run.smithi116.stdout: "services_complete": [ 2024-06-01T02:09:01.825 INFO:teuthology.orchestra.run.smithi116.stdout: "mon", 2024-06-01T02:09:01.826 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr" 2024-06-01T02:09:01.826 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-06-01T02:09:01.826 INFO:teuthology.orchestra.run.smithi116.stdout: "progress": "9/23 daemons upgraded", 2024-06-01T02:09:01.826 INFO:teuthology.orchestra.run.smithi116.stdout: "message": "Currently upgrading osd daemons", 2024-06-01T02:09:01.826 INFO:teuthology.orchestra.run.smithi116.stdout: "is_paused": false 2024-06-01T02:09:01.826 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:09:02.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:01 smithi161 ceph-mon[84406]: pgmap v176: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:09:02.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:01 smithi161 ceph-mon[84406]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-01T02:09:02.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:01 smithi161 ceph-mon[84406]: Upgrade: osd.4 is safe to restart 2024-06-01T02:09:02.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:01 smithi161 ceph-mon[84406]: from='client.64348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:09:02.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:01 smithi161 ceph-mon[84406]: from='client.64354 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:09:02.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:01 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:02.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:01 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:09:02.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:01 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:02.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:01 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/523488524' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:02.217 INFO:teuthology.orchestra.run.smithi116.stdout:HEALTH_OK 2024-06-01T02:09:02.217 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:01 smithi116 ceph-mon[100606]: pgmap v176: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:09:02.218 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:01 smithi116 ceph-mon[100606]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-01T02:09:02.218 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:01 smithi116 ceph-mon[100606]: Upgrade: osd.4 is safe to restart 2024-06-01T02:09:02.218 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:01 smithi116 ceph-mon[100606]: from='client.64348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:09:02.218 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:01 smithi116 ceph-mon[100606]: from='client.64354 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:09:02.218 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:01 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:02.218 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:01 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:09:02.218 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:01 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:02.218 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:01 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/523488524' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:02.218 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:01 smithi116 ceph-mon[89502]: pgmap v176: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:09:02.218 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:01 smithi116 ceph-mon[89502]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-01T02:09:02.219 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:01 smithi116 ceph-mon[89502]: Upgrade: osd.4 is safe to restart 2024-06-01T02:09:02.219 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:01 smithi116 ceph-mon[89502]: from='client.64348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:09:02.219 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:01 smithi116 ceph-mon[89502]: from='client.64354 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:09:02.219 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:02.219 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:09:02.219 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:02.219 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:01 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/523488524' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:02.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:02 smithi161 ceph-mon[84406]: from='client.64360 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:09:02.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:02 smithi161 ceph-mon[84406]: from='client.44406 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:09:02.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:02 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/938216455' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:09:02.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:02 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:02.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:02 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-01T02:09:02.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:02 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:03.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:02 smithi116 ceph-mon[89502]: from='client.64360 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:09:03.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:02 smithi116 ceph-mon[89502]: from='client.44406 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:09:03.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:02 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/938216455' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:09:03.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:03.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-01T02:09:03.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:03.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:02 smithi116 ceph-mon[100606]: from='client.64360 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:09:03.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:02 smithi116 ceph-mon[100606]: from='client.44406 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:09:03.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:02 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/938216455' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:09:03.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:02 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:03.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:02 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-01T02:09:03.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:02 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:04.118 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:03 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:09:04.119 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:03 smithi161 ceph-mon[84406]: pgmap v177: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:09:04.119 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:03 smithi161 ceph-mon[84406]: Upgrade: Updating osd.4 2024-06-01T02:09:04.119 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:03 smithi161 ceph-mon[84406]: Deploying daemon osd.4 on smithi161 2024-06-01T02:09:04.122 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:03 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:09:04.123 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:03 smithi116 ceph-mon[89502]: pgmap v177: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:09:04.123 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:03 smithi116 ceph-mon[89502]: Upgrade: Updating osd.4 2024-06-01T02:09:04.123 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:03 smithi116 ceph-mon[89502]: Deploying daemon osd.4 on smithi161 2024-06-01T02:09:04.123 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:03 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:09:04.123 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:03 smithi116 ceph-mon[100606]: pgmap v177: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:09:04.123 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:03 smithi116 ceph-mon[100606]: Upgrade: Updating osd.4 2024-06-01T02:09:04.123 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:03 smithi116 ceph-mon[100606]: Deploying daemon osd.4 on smithi161 2024-06-01T02:09:06.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:05 smithi116 ceph-mon[89502]: pgmap v178: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 341 B/s rd, 0 op/s 2024-06-01T02:09:06.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:05 smithi116 ceph-mon[100606]: pgmap v178: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 341 B/s rd, 0 op/s 2024-06-01T02:09:06.149 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:05 smithi161 ceph-mon[84406]: pgmap v178: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 341 B/s rd, 0 op/s 2024-06-01T02:09:07.071 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:09:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:09:06] "GET /metrics HTTP/1.1" 200 38448 "" "Prometheus/2.33.4" 2024-06-01T02:09:07.675 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:07 smithi161 systemd[1]: Stopping Ceph osd.4 for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:09:07.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:07 smithi161 ceph-mon[84406]: pgmap v179: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 851 B/s rd, 0 op/s 2024-06-01T02:09:07.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:07 smithi161 ceph-mon[84406]: osd.4 marked itself down and dead 2024-06-01T02:09:07.944 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[30761]: 2024-06-01T02:09:07.679+0000 7fc4bc8d9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-01T02:09:07.945 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[30761]: 2024-06-01T02:09:07.679+0000 7fc4bc8d9700 -1 osd.4 148 *** Got signal Terminated *** 2024-06-01T02:09:07.945 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[30761]: 2024-06-01T02:09:07.679+0000 7fc4bc8d9700 -1 osd.4 148 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-01T02:09:08.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:07 smithi116 ceph-mon[100606]: pgmap v179: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 851 B/s rd, 0 op/s 2024-06-01T02:09:08.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:07 smithi116 ceph-mon[100606]: osd.4 marked itself down and dead 2024-06-01T02:09:08.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:07 smithi116 ceph-mon[89502]: pgmap v179: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 851 B/s rd, 0 op/s 2024-06-01T02:09:08.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:07 smithi116 ceph-mon[89502]: osd.4 marked itself down and dead 2024-06-01T02:09:08.625 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:08 smithi161 podman[96939]: 2024-06-01 02:09:08.333068583 +0000 UTC m=+0.751244213 container died 70b3f89b0bd9b1406993cc270acf85ab40ae972459ed5259da270429b6b0e6a6 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4, io.openshift.tags=base centos centos-stream, io.k8s.display-name=CentOS Stream 8, name=centos-stream, io.buildah.version=1.19.8, io.openshift.expose-services=, vendor=Red Hat, Inc., GIT_CLEAN=True, GIT_BRANCH=HEAD, vcs-type=git, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, ceph=True, build-date=2022-05-03T08:36:31.336870, version=8, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.component=centos-stream-container, GIT_REPO=https://github.com/ceph/ceph-container.git, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, architecture=x86_64, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, RELEASE=HEAD, distribution-scope=public, release=754, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-17.2.0, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb) 2024-06-01T02:09:08.877 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:08 smithi161 podman[96939]: 2024-06-01 02:09:08.629495728 +0000 UTC m=+1.047671358 container cleanup 70b3f89b0bd9b1406993cc270acf85ab40ae972459ed5259da270429b6b0e6a6 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-17.2.0, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, GIT_CLEAN=True, RELEASE=HEAD, vcs-type=git, io.openshift.tags=base centos centos-stream, com.redhat.component=centos-stream-container, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, io.openshift.expose-services=, distribution-scope=public, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.buildah.version=1.19.8, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, architecture=x86_64, release=754, vendor=Red Hat, Inc., vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, build-date=2022-05-03T08:36:31.336870, GIT_BRANCH=HEAD, name=centos-stream, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, io.k8s.display-name=CentOS Stream 8, version=8) 2024-06-01T02:09:08.877 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:08 smithi161 bash[96939]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4 2024-06-01T02:09:08.877 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:08 smithi161 podman[96956]: 2024-06-01 02:09:08.864220712 +0000 UTC m=+0.532138855 container remove 70b3f89b0bd9b1406993cc270acf85ab40ae972459ed5259da270429b6b0e6a6 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4, io.openshift.tags=base centos centos-stream, vcs-type=git, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, ceph=True, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., release=754, io.openshift.expose-services=, maintainer=Guillaume Abrioux , com.redhat.component=centos-stream-container, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, CEPH_POINT_RELEASE=-17.2.0, name=centos-stream, vendor=Red Hat, Inc., com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, GIT_BRANCH=HEAD, io.buildah.version=1.19.8, io.k8s.display-name=CentOS Stream 8, version=8, architecture=x86_64, GIT_CLEAN=True, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, GIT_REPO=https://github.com/ceph/ceph-container.git, distribution-scope=public, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, RELEASE=HEAD, build-date=2022-05-03T08:36:31.336870) 2024-06-01T02:09:09.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:08 smithi161 ceph-mon[84406]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:09:09.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:08 smithi161 ceph-mon[84406]: osdmap e149: 8 total, 7 up, 8 in 2024-06-01T02:09:09.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:08 smithi161 ceph-mon[84406]: osdmap e150: 8 total, 7 up, 8 in 2024-06-01T02:09:09.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:08 smithi116 ceph-mon[89502]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:09:09.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:08 smithi116 ceph-mon[89502]: osdmap e149: 8 total, 7 up, 8 in 2024-06-01T02:09:09.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:08 smithi116 ceph-mon[89502]: osdmap e150: 8 total, 7 up, 8 in 2024-06-01T02:09:09.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:08 smithi116 ceph-mon[100606]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:09:09.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:08 smithi116 ceph-mon[100606]: osdmap e149: 8 total, 7 up, 8 in 2024-06-01T02:09:09.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:08 smithi116 ceph-mon[100606]: osdmap e150: 8 total, 7 up, 8 in 2024-06-01T02:09:09.455 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:09 smithi161 podman[97042]: 2024-06-01 02:09:09.10769294 +0000 UTC m=+0.017751673 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:09:09.455 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:09 smithi161 podman[97042]: 2024-06-01 02:09:09.223687474 +0000 UTC m=+0.133746184 container create 7cfb5a2771277ebe40d362ed5b89575eeffc573c115e9656a861c7b989cc8e45 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, ceph=True) 2024-06-01T02:09:09.754 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:09 smithi161 podman[97042]: 2024-06-01 02:09:09.459732968 +0000 UTC m=+0.369791680 container init 7cfb5a2771277ebe40d362ed5b89575eeffc573c115e9656a861c7b989cc8e45 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-01T02:09:09.754 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:09 smithi161 podman[97042]: 2024-06-01 02:09:09.463368167 +0000 UTC m=+0.373426885 container start 7cfb5a2771277ebe40d362ed5b89575eeffc573c115e9656a861c7b989cc8e45 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, ceph=True) 2024-06-01T02:09:09.754 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:09 smithi161 podman[97042]: 2024-06-01 02:09:09.526071826 +0000 UTC m=+0.436130543 container attach 7cfb5a2771277ebe40d362ed5b89575eeffc573c115e9656a861c7b989cc8e45 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-06-01T02:09:10.049 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:09 smithi161 ceph-mon[84406]: pgmap v181: 161 pgs: 21 stale+active+clean, 140 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 1022 B/s rd, 0 op/s 2024-06-01T02:09:10.049 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:09 smithi161 podman[97042]: 2024-06-01 02:09:09.75958174 +0000 UTC m=+0.669640460 container died 7cfb5a2771277ebe40d362ed5b89575eeffc573c115e9656a861c7b989cc8e45 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, RELEASE=squid-3d89acc, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528) 2024-06-01T02:09:10.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:09 smithi116 ceph-mon[89502]: pgmap v181: 161 pgs: 21 stale+active+clean, 140 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 1022 B/s rd, 0 op/s 2024-06-01T02:09:10.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:09 smithi116 ceph-mon[100606]: pgmap v181: 161 pgs: 21 stale+active+clean, 140 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 1022 B/s rd, 0 op/s 2024-06-01T02:09:10.311 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:10 smithi161 podman[97069]: 2024-06-01 02:09:10.279819423 +0000 UTC m=+0.513804278 container remove 7cfb5a2771277ebe40d362ed5b89575eeffc573c115e9656a861c7b989cc8e45 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc) 2024-06-01T02:09:10.311 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:10 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.4.service: Deactivated successfully. 2024-06-01T02:09:10.311 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:10 smithi161 systemd[1]: Stopped Ceph osd.4 for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:09:10.311 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:10 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.4.service: Consumed 11.837s CPU time. 2024-06-01T02:09:10.694 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:10 smithi161 systemd[1]: Starting Ceph osd.4 for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:09:11.012 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:10 smithi161 podman[97171]: 2024-06-01 02:09:10.722452746 +0000 UTC m=+0.105633300 container create c256d1d8ab67441cb8269ea8674e55d712133128fd62bb4ee8927ca444fa8037 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-activate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-06-01T02:09:11.012 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:10 smithi161 podman[97171]: 2024-06-01 02:09:10.634278706 +0000 UTC m=+0.017459233 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:09:11.012 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:10 smithi161 podman[97171]: 2024-06-01 02:09:10.925242125 +0000 UTC m=+0.308422642 container init c256d1d8ab67441cb8269ea8674e55d712133128fd62bb4ee8927ca444fa8037 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-activate, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-06-01T02:09:11.012 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:10 smithi161 podman[97171]: 2024-06-01 02:09:10.928664699 +0000 UTC m=+0.311845208 container start c256d1d8ab67441cb8269ea8674e55d712133128fd62bb4ee8927ca444fa8037 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-activate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-01T02:09:11.012 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:11 smithi161 podman[97171]: 2024-06-01 02:09:11.016774056 +0000 UTC m=+0.399954596 container attach c256d1d8ab67441cb8269ea8674e55d712133128fd62bb4ee8927ca444fa8037 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-activate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240528) 2024-06-01T02:09:11.694 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:11 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-activate[97186]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-06-01T02:09:11.694 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:11 smithi161 bash[97171]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-06-01T02:09:11.695 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:11 smithi161 bash[97171]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-06-01T02:09:11.695 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:11 smithi161 bash[97171]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-06-01T02:09:11.695 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:11 smithi161 bash[97171]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-06-01T02:09:11.695 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:11 smithi161 bash[97171]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-4/block 2024-06-01T02:09:11.695 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:11 smithi161 bash[97171]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-06-01T02:09:11.695 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:11 smithi161 bash[97171]: --> ceph-volume raw activate successful for osd ID: 4 2024-06-01T02:09:11.695 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:11 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-activate[97186]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-06-01T02:09:11.695 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:11 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-activate[97186]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-06-01T02:09:11.695 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:11 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-activate[97186]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-06-01T02:09:11.695 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:11 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-activate[97186]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-4/block 2024-06-01T02:09:11.695 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:11 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-activate[97186]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-06-01T02:09:11.695 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:11 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-activate[97186]: --> ceph-volume raw activate successful for osd ID: 4 2024-06-01T02:09:11.695 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:11 smithi161 podman[97171]: 2024-06-01 02:09:11.406099342 +0000 UTC m=+0.789279851 container died c256d1d8ab67441cb8269ea8674e55d712133128fd62bb4ee8927ca444fa8037 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-activate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0) 2024-06-01T02:09:11.966 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:11 smithi161 ceph-mon[84406]: pgmap v183: 161 pgs: 21 stale+active+clean, 140 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:09:12.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:11 smithi116 ceph-mon[89502]: pgmap v183: 161 pgs: 21 stale+active+clean, 140 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:09:12.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:11 smithi116 ceph-mon[100606]: pgmap v183: 161 pgs: 21 stale+active+clean, 140 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:09:12.330 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:11 smithi161 podman[97318]: 2024-06-01 02:09:11.971060042 +0000 UTC m=+0.555858343 container remove c256d1d8ab67441cb8269ea8674e55d712133128fd62bb4ee8927ca444fa8037 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-activate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-01T02:09:12.330 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:12 smithi161 podman[97381]: 2024-06-01 02:09:12.087046033 +0000 UTC m=+0.017921488 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:09:12.331 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:12 smithi161 podman[97381]: 2024-06-01 02:09:12.213748382 +0000 UTC m=+0.144623849 container create 01df0d1562f0a495eb2424df79aa2f367443cc2d6c2af979ac870205e0e08e2f (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528) 2024-06-01T02:09:12.694 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:12 smithi161 podman[97381]: 2024-06-01 02:09:12.432807166 +0000 UTC m=+0.363682614 container init 01df0d1562f0a495eb2424df79aa2f367443cc2d6c2af979ac870205e0e08e2f (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-06-01T02:09:12.694 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:12 smithi161 podman[97381]: 2024-06-01 02:09:12.436120188 +0000 UTC m=+0.366995632 container start 01df0d1562f0a495eb2424df79aa2f367443cc2d6c2af979ac870205e0e08e2f (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-06-01T02:09:12.694 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:12 smithi161 ceph-osd[97400]: -- 172.21.15.161:0/3565553293 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55edd1699680 con 0x55edd1663400 2024-06-01T02:09:12.694 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:12 smithi161 bash[97381]: 01df0d1562f0a495eb2424df79aa2f367443cc2d6c2af979ac870205e0e08e2f 2024-06-01T02:09:12.694 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:12 smithi161 systemd[1]: Started Ceph osd.4 for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:09:13.444 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:12 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[97396]: 2024-06-01T02:09:12.977+0000 7f668dfd97c0 -1 Falling back to public interface 2024-06-01T02:09:13.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:09:13.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: pgmap v184: 161 pgs: 14 active+undersized+degraded, 10 stale+active+clean, 19 active+undersized, 118 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 638 B/s rd, 0 op/s; 54/726 objects degraded (7.438%) 2024-06-01T02:09:13.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:13.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:13.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:13.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:13.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:13.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:13.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:13.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:13.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:13.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: Health check failed: Degraded data redundancy: 54/726 objects degraded (7.438%), 14 pgs degraded (PG_DEGRADED) 2024-06-01T02:09:13.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:13.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:13.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:13.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:13.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:13.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:13.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:13.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:13.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:09:13.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: pgmap v184: 161 pgs: 14 active+undersized+degraded, 10 stale+active+clean, 19 active+undersized, 118 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 638 B/s rd, 0 op/s; 54/726 objects degraded (7.438%) 2024-06-01T02:09:13.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:13.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:13.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:13.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:13.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:13.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:13.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:13.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:13.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:13.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: Health check failed: Degraded data redundancy: 54/726 objects degraded (7.438%), 14 pgs degraded (PG_DEGRADED) 2024-06-01T02:09:13.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:13.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:13.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:13.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:13.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:13.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:13.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:13.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:13.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:13.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:13.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:13.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:13.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:13.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:13.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:13.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:13.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:13.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:13.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:13.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:13.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:13.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:13.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:13.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:13.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:13.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:13.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:13.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:13.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:13.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:13.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:13.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:13.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:13.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:13.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:13.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:13.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:13.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:13.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:13.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:13.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:09:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: pgmap v184: 161 pgs: 14 active+undersized+degraded, 10 stale+active+clean, 19 active+undersized, 118 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 638 B/s rd, 0 op/s; 54/726 objects degraded (7.438%) 2024-06-01T02:09:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:13.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:13.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:13.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:13.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:13.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:13.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:13.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: Health check failed: Degraded data redundancy: 54/726 objects degraded (7.438%), 14 pgs degraded (PG_DEGRADED) 2024-06-01T02:09:13.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:13.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:13.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:13.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:13.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:13.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:13.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:13.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:13.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:13.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:13.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:13.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:13.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:13.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:13.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:13.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:13.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:13.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:13.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:13.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:13.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:13.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:13.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:13.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:13.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:13.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:14.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:14.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:14.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:14.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:14.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:14.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:14.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:14.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:14.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:14.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:14.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:14.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:14.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:14.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:14.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:14.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:14.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:14.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:14.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:14.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:14.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:14.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:14.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:14.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:14.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:14.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:14.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:14.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:14.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:14.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:14.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:14.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:14.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:14.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:14.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:14.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:14.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:14.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:14.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:14.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:14.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:14.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:14.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:14.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:14.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:14.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:14.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:14.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:14.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:14.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:14.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:14.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:14.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:14.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:14.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:14.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:14.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:14.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:14.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:14.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:14.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:14.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:14.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:14.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:14.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:14.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:14.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:14.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:14.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:14.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:14.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:14.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:14.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:14.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:14.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:14.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:14.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:14.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:14.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:14.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:14.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:14.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:14.804 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:14.804 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:14.804 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:14.804 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:14.804 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:14.804 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:14.804 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:14.804 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:14.805 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:14.805 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:14.805 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:14.805 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:14.805 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:14.805 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:14.805 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:14.806 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:14.806 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:14.806 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:14.806 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:14.806 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:14.806 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:14.807 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:14.807 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:14.807 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:14.828 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:14.828 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:14.828 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:14.828 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:14.828 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:14.829 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:14.829 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:14.829 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:14.829 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:14.829 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:14.829 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:14.829 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:14.829 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:14.829 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:14.829 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:14.830 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:14.830 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:14.830 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:14.830 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:14.830 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:14.830 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:14.830 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:14.830 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:14.830 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:14.830 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:14.831 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:14.831 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:14.831 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:14.831 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:14.831 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:14.831 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:14.831 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:14.831 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:14.831 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:14.831 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:14.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:14.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:14.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:14.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:14.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:14.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:14.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:14.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:14.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:14.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:14.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:14.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:14.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:14.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:14.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:14.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:14.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:14.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:15.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:15.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:15.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: pgmap v185: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 98/726 objects degraded (13.499%) 2024-06-01T02:09:15.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:15.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:15.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:15.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:15.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:15.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:15.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:15.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:15.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:15.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:15.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:15.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:15.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:15.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:15.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:15.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:15.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:15.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:15.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:15.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:15.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:15.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:15.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:15.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:15.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:15.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:15.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:15.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:15.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:15.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:15.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:15.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:15.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:15.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:15.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:15.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:15.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:15.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:15.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:15.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:15.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:15.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:15.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:16.032 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[97396]: 2024-06-01T02:09:15.737+0000 7f668dfd97c0 -1 osd.4 0 read_superblock omap replica is missing. 2024-06-01T02:09:16.032 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:15 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[97396]: 2024-06-01T02:09:15.772+0000 7f668dfd97c0 -1 osd.4 148 log_to_monitors true 2024-06-01T02:09:16.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:16.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:16.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: pgmap v185: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 98/726 objects degraded (13.499%) 2024-06-01T02:09:16.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:16.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:16.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:16.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:16.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:16.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:16.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:16.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:16.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:16.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:16.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:16.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:16.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:16.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:16.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:16.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:16.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:16.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:16.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:16.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:16.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:16.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:16.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:16.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:16.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:16.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:16.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:16.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:16.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: pgmap v185: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 98/726 objects degraded (13.499%) 2024-06-01T02:09:16.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:16.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:16.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:16.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:16.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:16.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:16.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:16.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:16.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:16.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:16.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:16.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:16.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:16.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:16.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:16.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:16.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:16.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:16.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:16.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:16.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:16.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:16.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:16.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:16.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:16.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:16.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:16.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:16.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:16.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:16.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:16.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:16.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:16.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:16.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:16.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:16.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:16.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:16.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:16.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:16.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:16.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:16.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:16.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:16.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:16.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:16.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:16.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:16.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:16.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:16.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:16.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:16.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:16.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:16.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:16.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:16.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:16.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:16.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:16.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:16.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:16.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:16.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:16.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:16.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:16.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:16.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:16.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:16.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:16.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:16.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:16.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:16.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:16.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:16.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:16.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='osd.4 [v2:172.21.15.161:6800/3733197228,v1:172.21.15.161:6801/3733197228]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-01T02:09:16.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-01T02:09:16.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:16.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:16.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:16.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:16.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:09:16.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:16.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:16.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:16.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:16.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:16.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:16.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:16.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:16.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:16.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:16.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:16.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:16.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:16.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:16.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:16.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:16.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:16.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:16.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:16.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:16.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:16.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:16.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:17.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:17.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:17.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:17.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:17.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:17.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:17.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:17.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:17.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:17.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:17.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:17.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:17.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:17.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:17.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:17.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='osd.4 [v2:172.21.15.161:6800/3733197228,v1:172.21.15.161:6801/3733197228]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-01T02:09:17.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-01T02:09:17.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:17.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:17.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:17.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:17.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:09:17.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:17.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:17.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:17.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:17.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:17.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:17.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:17.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:17.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:17.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:17.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:17.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:17.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:17.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:17.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:17.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:17.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:17.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:17.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:17.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:17.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:17.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='osd.4 [v2:172.21.15.161:6800/3733197228,v1:172.21.15.161:6801/3733197228]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-01T02:09:17.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-01T02:09:17.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:17.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:17.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:17.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:17.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:09:17.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:17.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:17.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:17.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:17.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:17.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:17.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:17.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:17.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:17.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:17.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:17.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:17.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:17.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:17.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:17.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:17.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:17.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:17.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:17.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:17.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:17.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:17.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:17.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:17.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:17.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:17.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:17.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:17.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:17.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:17.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:17.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:17.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:17.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:17.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:17.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:17.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:17.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:17.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:17.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:17.073 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:09:16 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:09:16] "GET /metrics HTTP/1.1" 200 38458 "" "Prometheus/2.33.4" 2024-06-01T02:09:17.444 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[97396]: 2024-06-01T02:09:17.192+0000 7f667b428700 -1 osd.4 148 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-01T02:09:17.766 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:17.766 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:17.766 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: pgmap v186: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 98/726 objects degraded (13.499%) 2024-06-01T02:09:17.766 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:17.766 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:17.767 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:17.767 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:17.767 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:17.767 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:17.767 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:17.767 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:17.767 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:17.767 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:17.767 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:17.767 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:17.767 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:17.767 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:17.767 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-01T02:09:17.768 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='osd.4 [v2:172.21.15.161:6800/3733197228,v1:172.21.15.161:6801/3733197228]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:09:17.768 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: osdmap e151: 8 total, 7 up, 8 in 2024-06-01T02:09:17.768 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:09:17.768 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:17.768 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:17.768 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:17.768 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:17.768 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:17.768 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:17.768 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:17.768 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:17.768 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:17.769 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:17.769 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:17.769 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='osd.4 ' entity='osd.4' 2024-06-01T02:09:17.769 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:17.769 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:17.769 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:17.769 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:17.769 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:17.769 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:17.769 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:17.769 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:17.769 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:17.769 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:17.769 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:17.770 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:17.770 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:17.770 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:17.770 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: pgmap v186: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 98/726 objects degraded (13.499%) 2024-06-01T02:09:18.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-01T02:09:18.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='osd.4 [v2:172.21.15.161:6800/3733197228,v1:172.21.15.161:6801/3733197228]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:09:18.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: osdmap e151: 8 total, 7 up, 8 in 2024-06-01T02:09:18.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:09:18.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: pgmap v186: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 98/726 objects degraded (13.499%) 2024-06-01T02:09:18.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-01T02:09:18.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='osd.4 [v2:172.21.15.161:6800/3733197228,v1:172.21.15.161:6801/3733197228]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:09:18.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: osdmap e151: 8 total, 7 up, 8 in 2024-06-01T02:09:18.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:09:18.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.051 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='osd.4 ' entity='osd.4' 2024-06-01T02:09:18.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='osd.4 ' entity='osd.4' 2024-06-01T02:09:18.053 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.054 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.054 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.054 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.054 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.054 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.054 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.054 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.054 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.054 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.054 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.054 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.054 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.054 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.054 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.676 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.676 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.676 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.676 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.676 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.676 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.676 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.676 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.676 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.677 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.677 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.677 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.677 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.677 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.677 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.677 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.677 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.677 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.677 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.677 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.677 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.677 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.677 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.678 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.678 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.678 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.678 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.678 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.678 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.678 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.678 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.678 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.679 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.679 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:09:18.679 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.679 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: osd.4 [v2:172.21.15.161:6800/3733197228,v1:172.21.15.161:6801/3733197228] boot 2024-06-01T02:09:18.679 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: osdmap e152: 8 total, 8 up, 8 in 2024-06-01T02:09:18.679 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T02:09:18.679 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.679 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.679 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.679 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.679 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.679 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.679 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.679 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.680 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.680 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.680 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.680 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.680 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.680 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.944 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.944 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.944 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.945 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.945 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.945 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.945 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.945 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.945 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.945 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.945 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.945 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.945 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.945 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.945 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.945 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.945 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.946 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.946 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.946 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.946 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.946 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.946 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.946 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.946 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.946 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.946 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.946 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.946 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.947 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.947 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.947 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.947 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.947 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.947 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.947 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.947 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.947 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.947 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.947 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.948 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.948 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.948 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.948 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.948 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.948 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.948 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.948 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.948 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.948 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.948 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.948 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.948 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.948 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.948 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.949 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.949 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:18.949 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.949 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.949 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.949 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.949 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.949 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.949 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:09:18.949 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.950 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: osd.4 [v2:172.21.15.161:6800/3733197228,v1:172.21.15.161:6801/3733197228] boot 2024-06-01T02:09:18.950 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: osdmap e152: 8 total, 8 up, 8 in 2024-06-01T02:09:18.950 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T02:09:18.950 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.950 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.950 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.950 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.950 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.950 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.950 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.950 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.950 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.950 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.951 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.951 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.951 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.951 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.951 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.951 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.951 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.951 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:09:18.951 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.951 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: osd.4 [v2:172.21.15.161:6800/3733197228,v1:172.21.15.161:6801/3733197228] boot 2024-06-01T02:09:18.951 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: osdmap e152: 8 total, 8 up, 8 in 2024-06-01T02:09:18.951 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T02:09:18.952 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.952 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.952 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.952 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.952 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.952 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.952 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.952 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.952 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:18.952 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:18.952 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:18.952 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:18.952 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:18.952 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:19.831 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:19.831 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: pgmap v189: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 206 MiB used, 715 GiB / 715 GiB avail; 98/726 objects degraded (13.499%) 2024-06-01T02:09:19.831 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:19.831 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:19.831 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:19.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:19.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:19.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:19.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:19.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:19.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:19.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:19.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:19.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:19.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:19.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:19.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:19.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:19.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:19.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:19.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:19.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:19.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:19.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:19.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:19.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:19.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:19.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:19.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:19.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:19.834 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:19.834 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:19.834 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:19.834 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: osdmap e153: 8 total, 8 up, 8 in 2024-06-01T02:09:19.834 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:19.834 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:19.834 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:19.834 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:19.834 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:19.834 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:19.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:19.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:19.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:19.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:19.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:19.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:19.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:19.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:19.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:19.928 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:19.928 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: pgmap v189: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 206 MiB used, 715 GiB / 715 GiB avail; 98/726 objects degraded (13.499%) 2024-06-01T02:09:19.928 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:19.929 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:19.929 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:19.929 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:19.929 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:19.929 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:19.929 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:19.929 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:19.929 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:19.929 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:19.929 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:19.930 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:19.930 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:19.930 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:19.930 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:19.930 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:19.930 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:19.930 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:19.930 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:19.930 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:19.930 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:19.930 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:19.930 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:19.930 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:19.931 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:19.931 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:19.931 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:19.931 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:19.931 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: pgmap v189: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 206 MiB used, 715 GiB / 715 GiB avail; 98/726 objects degraded (13.499%) 2024-06-01T02:09:19.932 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:19.932 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:19.932 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:19.932 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:19.932 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:19.932 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:19.932 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:19.932 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:19.932 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:19.932 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:19.932 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:19.932 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:19.932 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:19.932 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:19.933 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:19.933 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:19.933 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:19.933 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:19.933 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:19.933 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:19.933 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:19.933 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:19.933 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:19.933 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:19.933 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:19.934 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:19.934 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:19.934 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:19.934 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:19.934 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:19.934 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:19.934 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: osdmap e153: 8 total, 8 up, 8 in 2024-06-01T02:09:19.934 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:19.934 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:19.934 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:19.935 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:19.935 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:19.935 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:19.935 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:19.935 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:19.935 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:19.935 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:19.935 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:19.935 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:19.935 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:19.935 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:19.935 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:19.936 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:19.936 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:19.936 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:19.936 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:19.936 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: osdmap e153: 8 total, 8 up, 8 in 2024-06-01T02:09:19.936 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:19.936 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:19.936 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:19.936 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:19.936 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:19.936 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:19.936 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:19.936 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:19.937 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:19.937 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:19.937 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:19.937 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:19.937 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:19.937 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:19.937 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:20.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:20.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:20.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:20.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:20.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:20.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:20.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:20.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:20.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:20.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:20.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:20.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:20.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:20.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:20.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:20.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:20.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:20.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:20.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:20.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:20.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:20.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:20.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:20.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:20.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:20.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:20.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:20.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:20.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:20.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:20.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:20.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:20.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:20.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:20.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:20.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:20.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:20.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:20.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:20.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:20.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:20.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:20.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:20.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:20.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:20.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:20.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:20.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:20.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:20.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:20.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:20.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:20.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.964 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:20.965 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:20.965 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:20.965 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:20.965 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:20.965 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.965 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:20.965 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:20.965 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:20.965 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:20.965 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:20.965 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.965 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:20.965 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:20.966 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:20.966 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:20.966 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:20.966 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.966 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:20.966 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:20.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:20.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:20.966 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:20.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:20.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:20.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:20.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:20.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:20.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:20.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:20.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:20.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:20.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:20.967 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:20.968 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.968 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:20.968 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:20.968 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:20.968 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:20.968 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:20.968 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.968 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:20.968 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:20.968 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:20.968 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:20.968 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:20.968 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.968 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:20.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:20.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:20.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:20.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:20.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:20.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:20.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:20.969 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:20.970 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:20.970 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:20.970 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.970 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:20.970 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:20.970 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:20.970 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:20.970 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.970 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.970 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:20.970 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:20.970 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.970 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:20.971 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:20.971 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.971 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:20.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:20.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:20.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:20.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:20.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:20.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:20.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.971 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:20.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:20.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:20.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:20.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:20.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:20.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:20.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:20.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:20.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:20.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:20.972 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:20.973 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:20.973 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:21.690 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:21 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:21.690 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:21 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:21.690 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:21 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:21.690 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:21 smithi161 ceph-mon[84406]: pgmap v191: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 206 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 98/726 objects degraded (13.499%) 2024-06-01T02:09:21.690 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:21 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:21.690 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:21 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:21.690 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:21.690 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:21.691 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:21.691 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:21.691 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:21 smithi161 ceph-mon[84406]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-01T02:09:21.691 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-01T02:09:21.691 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:21 smithi161 ceph-mon[84406]: Upgrade: osd.4 is safe to restart 2024-06-01T02:09:21.691 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:21 smithi161 ceph-mon[84406]: Health check update: Degraded data redundancy: 98/726 objects degraded (13.499%), 24 pgs degraded (PG_DEGRADED) 2024-06-01T02:09:22.017 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:21 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:22.017 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:21 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:22.017 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:21 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:22.017 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:21 smithi116 ceph-mon[100606]: pgmap v191: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 206 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 98/726 objects degraded (13.499%) 2024-06-01T02:09:22.017 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:21 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:22.017 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:21 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:22.017 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:22.017 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:22.018 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:22.018 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:22.018 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:21 smithi116 ceph-mon[100606]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-01T02:09:22.018 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-01T02:09:22.018 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:21 smithi116 ceph-mon[100606]: Upgrade: osd.4 is safe to restart 2024-06-01T02:09:22.018 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:21 smithi116 ceph-mon[100606]: Health check update: Degraded data redundancy: 98/726 objects degraded (13.499%), 24 pgs degraded (PG_DEGRADED) 2024-06-01T02:09:22.018 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:21 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:22.018 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:21 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:22.018 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:21 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:22.018 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:21 smithi116 ceph-mon[89502]: pgmap v191: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 206 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 98/726 objects degraded (13.499%) 2024-06-01T02:09:22.019 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:21 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:22.019 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:21 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:22.019 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:22.019 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:22.019 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:22.019 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:22.019 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:21 smithi116 ceph-mon[89502]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-01T02:09:22.019 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-01T02:09:22.019 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:21 smithi116 ceph-mon[89502]: Upgrade: osd.4 is safe to restart 2024-06-01T02:09:22.019 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:21 smithi116 ceph-mon[89502]: Health check update: Degraded data redundancy: 98/726 objects degraded (13.499%), 24 pgs degraded (PG_DEGRADED) 2024-06-01T02:09:23.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:23 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:09:23.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:23 smithi161 ceph-mon[84406]: pgmap v192: 161 pgs: 6 active+undersized+degraded, 14 active+undersized, 141 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 34/726 objects degraded (4.683%) 2024-06-01T02:09:24.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:23 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:09:24.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:23 smithi116 ceph-mon[89502]: pgmap v192: 161 pgs: 6 active+undersized+degraded, 14 active+undersized, 141 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 34/726 objects degraded (4.683%) 2024-06-01T02:09:24.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:23 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:09:24.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:23 smithi116 ceph-mon[100606]: pgmap v192: 161 pgs: 6 active+undersized+degraded, 14 active+undersized, 141 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 34/726 objects degraded (4.683%) 2024-06-01T02:09:24.595 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-01T02:09:24.595 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:25.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-01T02:09:25.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:25.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-01T02:09:25.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:25.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:25 smithi161 ceph-mon[84406]: pgmap v193: 161 pgs: 161 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-01T02:09:25.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:25 smithi161 ceph-mon[84406]: Upgrade: Updating osd.4 2024-06-01T02:09:25.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:25 smithi161 ceph-mon[84406]: Deploying daemon osd.4 on smithi161 2024-06-01T02:09:25.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:25 smithi161 ceph-mon[84406]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 34/726 objects degraded (4.683%), 6 pgs degraded) 2024-06-01T02:09:25.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:25 smithi161 ceph-mon[84406]: Cluster is now healthy 2024-06-01T02:09:26.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:25 smithi116 ceph-mon[89502]: pgmap v193: 161 pgs: 161 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-01T02:09:26.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:25 smithi116 ceph-mon[89502]: Upgrade: Updating osd.4 2024-06-01T02:09:26.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:25 smithi116 ceph-mon[89502]: Deploying daemon osd.4 on smithi161 2024-06-01T02:09:26.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:25 smithi116 ceph-mon[89502]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 34/726 objects degraded (4.683%), 6 pgs degraded) 2024-06-01T02:09:26.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:25 smithi116 ceph-mon[89502]: Cluster is now healthy 2024-06-01T02:09:26.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:25 smithi116 ceph-mon[100606]: pgmap v193: 161 pgs: 161 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-01T02:09:26.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:25 smithi116 ceph-mon[100606]: Upgrade: Updating osd.4 2024-06-01T02:09:26.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:25 smithi116 ceph-mon[100606]: Deploying daemon osd.4 on smithi161 2024-06-01T02:09:26.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:25 smithi116 ceph-mon[100606]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 34/726 objects degraded (4.683%), 6 pgs degraded) 2024-06-01T02:09:26.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:25 smithi116 ceph-mon[100606]: Cluster is now healthy 2024-06-01T02:09:26.944 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:09:26 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:09:26] "GET /metrics HTTP/1.1" 200 38458 "" "Prometheus/2.33.4" 2024-06-01T02:09:27.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:27 smithi161 ceph-mon[84406]: pgmap v194: 161 pgs: 161 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 638 B/s rd, 0 op/s 2024-06-01T02:09:27.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:27.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:27.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:27.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:27.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:27.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:28.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:27 smithi116 ceph-mon[89502]: pgmap v194: 161 pgs: 161 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 638 B/s rd, 0 op/s 2024-06-01T02:09:28.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:28.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:28.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:28.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:28.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:28.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:28.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:27 smithi116 ceph-mon[100606]: pgmap v194: 161 pgs: 161 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 638 B/s rd, 0 op/s 2024-06-01T02:09:28.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:28.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:28.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:28.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:28.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:28.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:28.444 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:28 smithi161 systemd[1]: Stopping Ceph osd.4 for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:09:28.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:28 smithi161 ceph-mon[84406]: osd.4 marked itself down and dead 2024-06-01T02:09:28.944 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:28 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[97396]: 2024-06-01T02:09:28.448+0000 7f668463f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-01T02:09:28.944 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:28 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[97396]: 2024-06-01T02:09:28.448+0000 7f668463f700 -1 osd.4 153 *** Got signal Terminated *** 2024-06-01T02:09:28.944 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:28 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[97396]: 2024-06-01T02:09:28.448+0000 7f668463f700 -1 osd.4 153 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-01T02:09:29.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:28 smithi116 ceph-mon[89502]: osd.4 marked itself down and dead 2024-06-01T02:09:29.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:28 smithi116 ceph-mon[100606]: osd.4 marked itself down and dead 2024-06-01T02:09:29.345 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:29 smithi161 podman[101276]: 2024-06-01 02:09:29.031756106 +0000 UTC m=+0.678064712 container died 01df0d1562f0a495eb2424df79aa2f367443cc2d6c2af979ac870205e0e08e2f (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-06-01T02:09:29.665 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:29 smithi161 ceph-mon[84406]: pgmap v195: 161 pgs: 161 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:09:29.665 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:29 smithi161 ceph-mon[84406]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:09:29.665 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:29 smithi161 ceph-mon[84406]: osdmap e154: 8 total, 7 up, 8 in 2024-06-01T02:09:29.665 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:29 smithi161 podman[101276]: 2024-06-01 02:09:29.435225609 +0000 UTC m=+1.081534215 container cleanup 01df0d1562f0a495eb2424df79aa2f367443cc2d6c2af979ac870205e0e08e2f (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-06-01T02:09:29.665 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:29 smithi161 bash[101276]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4 2024-06-01T02:09:29.944 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:29 smithi161 podman[101291]: 2024-06-01 02:09:29.670065651 +0000 UTC m=+0.642024062 container remove 01df0d1562f0a495eb2424df79aa2f367443cc2d6c2af979ac870205e0e08e2f (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-06-01T02:09:30.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:29 smithi116 ceph-mon[89502]: pgmap v195: 161 pgs: 161 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:09:30.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:29 smithi116 ceph-mon[89502]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:09:30.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:29 smithi116 ceph-mon[89502]: osdmap e154: 8 total, 7 up, 8 in 2024-06-01T02:09:30.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:29 smithi116 ceph-mon[100606]: pgmap v195: 161 pgs: 161 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:09:30.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:29 smithi116 ceph-mon[100606]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:09:30.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:29 smithi116 ceph-mon[100606]: osdmap e154: 8 total, 7 up, 8 in 2024-06-01T02:09:30.260 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:30 smithi161 podman[101385]: 2024-06-01 02:09:29.915144402 +0000 UTC m=+0.017785010 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:09:30.261 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:30 smithi161 podman[101385]: 2024-06-01 02:09:30.037953575 +0000 UTC m=+0.140594174 container create 69adbb95a41a646a2c53b78bf6dd43180279618043603d103e55c4770743ca71 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-01T02:09:30.574 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:30 smithi161 podman[101385]: 2024-06-01 02:09:30.265400953 +0000 UTC m=+0.368041546 container init 69adbb95a41a646a2c53b78bf6dd43180279618043603d103e55c4770743ca71 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-01T02:09:30.575 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:30 smithi161 podman[101385]: 2024-06-01 02:09:30.269371582 +0000 UTC m=+0.372012183 container start 69adbb95a41a646a2c53b78bf6dd43180279618043603d103e55c4770743ca71 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , ceph=True) 2024-06-01T02:09:30.575 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:30 smithi161 podman[101385]: 2024-06-01 02:09:30.340285471 +0000 UTC m=+0.442926081 container attach 69adbb95a41a646a2c53b78bf6dd43180279618043603d103e55c4770743ca71 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, ceph=True) 2024-06-01T02:09:30.864 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:30 smithi161 podman[101385]: 2024-06-01 02:09:30.57915546 +0000 UTC m=+0.681796079 container died 69adbb95a41a646a2c53b78bf6dd43180279618043603d103e55c4770743ca71 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-deactivate, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-06-01T02:09:30.864 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:30 smithi161 ceph-mon[84406]: osdmap e155: 8 total, 7 up, 8 in 2024-06-01T02:09:31.042 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:30 smithi116 ceph-mon[100606]: osdmap e155: 8 total, 7 up, 8 in 2024-06-01T02:09:31.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:30 smithi116 ceph-mon[89502]: osdmap e155: 8 total, 7 up, 8 in 2024-06-01T02:09:31.694 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:31 smithi161 podman[101410]: 2024-06-01 02:09:31.353751734 +0000 UTC m=+0.762527648 container remove 69adbb95a41a646a2c53b78bf6dd43180279618043603d103e55c4770743ca71 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True) 2024-06-01T02:09:31.694 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:31 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.4.service: Deactivated successfully. 2024-06-01T02:09:31.694 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:31 smithi161 systemd[1]: Stopped Ceph osd.4 for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:09:31.694 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:31 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.4.service: Consumed 2.061s CPU time. 2024-06-01T02:09:31.695 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:31 smithi161 systemd[1]: Starting Ceph osd.4 for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:09:32.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:31 smithi161 ceph-mon[84406]: pgmap v198: 161 pgs: 21 stale+active+clean, 140 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:09:32.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:32.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:09:32.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:32.095 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:31 smithi161 podman[101509]: 2024-06-01 02:09:31.697869506 +0000 UTC m=+0.017514608 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:09:32.095 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:31 smithi161 podman[101509]: 2024-06-01 02:09:31.795386037 +0000 UTC m=+0.115031119 container create a6fb3e5577f82f712c96356e74ff7a8050aad8870495d54cc799d91a9decec5e (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-activate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True) 2024-06-01T02:09:32.095 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:32 smithi161 podman[101509]: 2024-06-01 02:09:32.006564769 +0000 UTC m=+0.326209866 container init a6fb3e5577f82f712c96356e74ff7a8050aad8870495d54cc799d91a9decec5e (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-activate, GIT_CLEAN=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-01T02:09:32.095 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:32 smithi161 podman[101509]: 2024-06-01 02:09:32.010142402 +0000 UTC m=+0.329787486 container start a6fb3e5577f82f712c96356e74ff7a8050aad8870495d54cc799d91a9decec5e (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-activate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True) 2024-06-01T02:09:32.095 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:32 smithi161 podman[101509]: 2024-06-01 02:09:32.097949528 +0000 UTC m=+0.417594625 container attach a6fb3e5577f82f712c96356e74ff7a8050aad8870495d54cc799d91a9decec5e (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-activate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-01T02:09:32.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:31 smithi116 ceph-mon[89502]: pgmap v198: 161 pgs: 21 stale+active+clean, 140 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:09:32.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:32.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:09:32.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:32.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:31 smithi116 ceph-mon[100606]: pgmap v198: 161 pgs: 21 stale+active+clean, 140 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:09:32.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:32.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:09:32.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:32.579 INFO:teuthology.orchestra.run.smithi116.stdout:true 2024-06-01T02:09:32.794 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:32 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-activate[101524]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-06-01T02:09:32.794 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:32 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-activate[101524]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-06-01T02:09:32.794 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:32 smithi161 bash[101509]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-06-01T02:09:32.794 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:32 smithi161 bash[101509]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-06-01T02:09:32.794 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:32 smithi161 bash[101509]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-06-01T02:09:32.794 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:32 smithi161 bash[101509]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-06-01T02:09:32.795 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:32 smithi161 bash[101509]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-4/block 2024-06-01T02:09:32.795 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:32 smithi161 bash[101509]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-06-01T02:09:32.795 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:32 smithi161 bash[101509]: --> ceph-volume raw activate successful for osd ID: 4 2024-06-01T02:09:32.795 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:32 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-activate[101524]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-06-01T02:09:32.795 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:32 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-activate[101524]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-06-01T02:09:32.795 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:32 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-activate[101524]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-4/block 2024-06-01T02:09:32.795 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:32 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-activate[101524]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-06-01T02:09:32.795 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:32 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-activate[101524]: --> ceph-volume raw activate successful for osd ID: 4 2024-06-01T02:09:32.795 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:32 smithi161 podman[101509]: 2024-06-01 02:09:32.483688671 +0000 UTC m=+0.803333758 container died a6fb3e5577f82f712c96356e74ff7a8050aad8870495d54cc799d91a9decec5e (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-activate, org.label-schema.build-date=20240528, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-06-01T02:09:33.106 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.15", "id": [3, 2]}]: dispatch 2024-06-01T02:09:33.106 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.16", "id": [7, 4]}]: dispatch 2024-06-01T02:09:33.106 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [0, 4]}]: dispatch 2024-06-01T02:09:33.106 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.0", "id": [3, 4]}]: dispatch 2024-06-01T02:09:33.106 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2, 3, 4]}]: dispatch 2024-06-01T02:09:33.106 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2024-06-01T02:09:33.106 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.3"}]: dispatch 2024-06-01T02:09:33.107 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.e"}]: dispatch 2024-06-01T02:09:33.107 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.14"}]: dispatch 2024-06-01T02:09:33.107 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.1c"}]: dispatch 2024-06-01T02:09:33.107 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:33.107 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:33.107 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:33.244 INFO:teuthology.orchestra.run.smithi116.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-01T02:09:33.244 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi116 smithi116 running 0s ago 18m - - 2024-06-01T02:09:33.244 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi161 smithi161 running 5s ago 17m - - 2024-06-01T02:09:33.244 INFO:teuthology.orchestra.run.smithi116.stdout:alertmanager.a smithi116 *:9093,9094 running (7m) 0s ago 12m 21.6M - 0.23.0 ba2b418f427c e0c297162ded 2024-06-01T02:09:33.244 INFO:teuthology.orchestra.run.smithi116.stdout:grafana.a smithi161 *:3000 running (7m) 5s ago 12m 53.3M - 8.3.5 dad864ee21e9 00b0a78ee88e 2024-06-01T02:09:33.244 INFO:teuthology.orchestra.run.smithi116.stdout:iscsi.foo.smithi116.gjuxzb smithi116 running (6m) 0s ago 11m 58.8M - 3.5 e1d6a67b021e 0e191fe3e3bb 2024-06-01T02:09:33.245 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.x smithi161 *:8443,9283,8765 running (6m) 5s ago 17m 527M - 19.0.0-2946-g3d89accf b045ac545f82 2d7fbb85e5f7 2024-06-01T02:09:33.245 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.y smithi116 *:8443,9283,8765 running (5m) 0s ago 19m 455M - 19.0.0-2946-g3d89accf b045ac545f82 bc9627c27a57 2024-06-01T02:09:33.245 INFO:teuthology.orchestra.run.smithi116.stdout:mon.a smithi116 running (4m) 0s ago 19m 64.3M 2048M 19.0.0-2946-g3d89accf b045ac545f82 b5e9b6f873ff 2024-06-01T02:09:33.245 INFO:teuthology.orchestra.run.smithi116.stdout:mon.b smithi161 running (3m) 5s ago 17m 43.1M 2048M 19.0.0-2946-g3d89accf b045ac545f82 aaa08c03cfc8 2024-06-01T02:09:33.245 INFO:teuthology.orchestra.run.smithi116.stdout:mon.c smithi116 running (3m) 0s ago 17m 48.5M 2048M 19.0.0-2946-g3d89accf b045ac545f82 bd0fb0e603fa 2024-06-01T02:09:33.245 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.a smithi116 *:9100 running (7m) 0s ago 13m 17.0M - 1.3.1 1dbe0e931976 e347748d17bb 2024-06-01T02:09:33.245 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.b smithi161 *:9100 running (7m) 5s ago 13m 18.1M - 1.3.1 1dbe0e931976 6e66182f5a1a 2024-06-01T02:09:33.245 INFO:teuthology.orchestra.run.smithi116.stdout:osd.0 smithi116 running (2m) 0s ago 17m 48.7M 4096M 19.0.0-2946-g3d89accf b045ac545f82 b4313d121b9d 2024-06-01T02:09:33.245 INFO:teuthology.orchestra.run.smithi116.stdout:osd.1 smithi116 running (115s) 0s ago 16m 50.6M 4096M 19.0.0-2946-g3d89accf b045ac545f82 dccdbeac5aae 2024-06-01T02:09:33.245 INFO:teuthology.orchestra.run.smithi116.stdout:osd.2 smithi116 running (80s) 0s ago 16m 45.2M 4096M 19.0.0-2946-g3d89accf b045ac545f82 1ad95efda404 2024-06-01T02:09:33.245 INFO:teuthology.orchestra.run.smithi116.stdout:osd.3 smithi116 running (50s) 0s ago 15m 48.8M 4096M 19.0.0-2946-g3d89accf b045ac545f82 bd00ad2c2556 2024-06-01T02:09:33.245 INFO:teuthology.orchestra.run.smithi116.stdout:osd.4 smithi161 running (21s) 5s ago 15m 51.3M 4096M 19.0.0-2946-g3d89accf b045ac545f82 01df0d1562f0 2024-06-01T02:09:33.245 INFO:teuthology.orchestra.run.smithi116.stdout:osd.5 smithi161 running (15m) 5s ago 15m 59.6M 4096M 17.2.0 e1d6a67b021e 68358b78a25d 2024-06-01T02:09:33.246 INFO:teuthology.orchestra.run.smithi116.stdout:osd.6 smithi161 running (14m) 5s ago 14m 60.2M 4096M 17.2.0 e1d6a67b021e 50abeb0551e7 2024-06-01T02:09:33.246 INFO:teuthology.orchestra.run.smithi116.stdout:osd.7 smithi161 running (14m) 5s ago 14m 63.4M 4096M 17.2.0 e1d6a67b021e 6004e6983735 2024-06-01T02:09:33.246 INFO:teuthology.orchestra.run.smithi116.stdout:prometheus.a smithi161 *:9095 running (6m) 5s ago 13m 76.9M - 2.33.4 514e6a882f6e e977f6ca1d73 2024-06-01T02:09:33.246 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi116.vzjrxj smithi116 *:8000 running (11m) 0s ago 11m 77.2M - 17.2.0 e1d6a67b021e df4fe3c1dbc3 2024-06-01T02:09:33.246 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi161.ashqxz smithi161 *:8000 running (11m) 5s ago 11m 78.0M - 17.2.0 e1d6a67b021e 3eecc1f3e2d3 2024-06-01T02:09:33.246 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi116.hokbmf smithi116 *:80 running (11m) 0s ago 11m 77.6M - 17.2.0 e1d6a67b021e 10950ed75dbd 2024-06-01T02:09:33.246 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi161.axjlzw smithi161 *:80 running (11m) 5s ago 11m 76.8M - 17.2.0 e1d6a67b021e 0b5d5ce47772 2024-06-01T02:09:33.246 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.15", "id": [3, 2]}]: dispatch 2024-06-01T02:09:33.246 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.16", "id": [7, 4]}]: dispatch 2024-06-01T02:09:33.247 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [0, 4]}]: dispatch 2024-06-01T02:09:33.247 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.0", "id": [3, 4]}]: dispatch 2024-06-01T02:09:33.247 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2, 3, 4]}]: dispatch 2024-06-01T02:09:33.247 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2024-06-01T02:09:33.247 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.3"}]: dispatch 2024-06-01T02:09:33.247 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.e"}]: dispatch 2024-06-01T02:09:33.247 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.14"}]: dispatch 2024-06-01T02:09:33.247 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.1c"}]: dispatch 2024-06-01T02:09:33.247 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:33.247 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:33.247 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:33.248 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.15", "id": [3, 2]}]: dispatch 2024-06-01T02:09:33.248 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.16", "id": [7, 4]}]: dispatch 2024-06-01T02:09:33.248 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [0, 4]}]: dispatch 2024-06-01T02:09:33.248 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.0", "id": [3, 4]}]: dispatch 2024-06-01T02:09:33.248 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2, 3, 4]}]: dispatch 2024-06-01T02:09:33.248 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2024-06-01T02:09:33.248 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.3"}]: dispatch 2024-06-01T02:09:33.248 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.e"}]: dispatch 2024-06-01T02:09:33.248 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.14"}]: dispatch 2024-06-01T02:09:33.248 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.1c"}]: dispatch 2024-06-01T02:09:33.248 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:33.248 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:33.249 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:33.444 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:33 smithi161 podman[101640]: 2024-06-01 02:09:33.1108131 +0000 UTC m=+0.621691304 container remove a6fb3e5577f82f712c96356e74ff7a8050aad8870495d54cc799d91a9decec5e (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=) 2024-06-01T02:09:33.444 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:33 smithi161 podman[101700]: 2024-06-01 02:09:33.226152607 +0000 UTC m=+0.017312555 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:09:33.444 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:33 smithi161 podman[101700]: 2024-06-01 02:09:33.336963651 +0000 UTC m=+0.128123601 container create 0084b9edf7a05ef25aaa4c197befae6f687658690e7d27c1323933da3c744ca6 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4, GIT_CLEAN=True, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-01T02:09:33.635 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:09:33.635 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": { 2024-06-01T02:09:33.635 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 3 2024-06-01T02:09:33.635 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:09:33.636 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": { 2024-06-01T02:09:33.636 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-01T02:09:33.636 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:09:33.636 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": { 2024-06-01T02:09:33.636 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 3, 2024-06-01T02:09:33.636 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 4 2024-06-01T02:09:33.636 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:09:33.636 INFO:teuthology.orchestra.run.smithi116.stdout: "rgw": { 2024-06-01T02:09:33.636 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-06-01T02:09:33.636 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:09:33.636 INFO:teuthology.orchestra.run.smithi116.stdout: "overall": { 2024-06-01T02:09:33.636 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 7, 2024-06-01T02:09:33.636 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 9 2024-06-01T02:09:33.637 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T02:09:33.637 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:09:33.834 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:33 smithi161 podman[101700]: 2024-06-01 02:09:33.547406881 +0000 UTC m=+0.338566808 container init 0084b9edf7a05ef25aaa4c197befae6f687658690e7d27c1323933da3c744ca6 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, ceph=True) 2024-06-01T02:09:33.834 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:33 smithi161 podman[101700]: 2024-06-01 02:09:33.550689094 +0000 UTC m=+0.341849040 container start 0084b9edf7a05ef25aaa4c197befae6f687658690e7d27c1323933da3c744ca6 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-06-01T02:09:33.834 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:33 smithi161 ceph-osd[101719]: -- 172.21.15.161:0/916538608 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559caf5ab680 con 0x559caf575400 2024-06-01T02:09:33.834 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:33 smithi161 bash[101700]: 0084b9edf7a05ef25aaa4c197befae6f687658690e7d27c1323933da3c744ca6 2024-06-01T02:09:33.834 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:33 smithi161 systemd[1]: Started Ceph osd.4 for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:09:33.983 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:09:33.983 INFO:teuthology.orchestra.run.smithi116.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-01T02:09:33.983 INFO:teuthology.orchestra.run.smithi116.stdout: "in_progress": true, 2024-06-01T02:09:33.983 INFO:teuthology.orchestra.run.smithi116.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-01T02:09:33.983 INFO:teuthology.orchestra.run.smithi116.stdout: "services_complete": [ 2024-06-01T02:09:33.983 INFO:teuthology.orchestra.run.smithi116.stdout: "mon", 2024-06-01T02:09:33.983 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr" 2024-06-01T02:09:33.983 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-06-01T02:09:33.983 INFO:teuthology.orchestra.run.smithi116.stdout: "progress": "9/23 daemons upgraded", 2024-06-01T02:09:33.983 INFO:teuthology.orchestra.run.smithi116.stdout: "message": "Currently upgrading osd daemons", 2024-06-01T02:09:33.983 INFO:teuthology.orchestra.run.smithi116.stdout: "is_paused": false 2024-06-01T02:09:33.983 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:09:33.984 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:09:33.984 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[89502]: pgmap v199: 161 pgs: 14 active+undersized+degraded, 10 stale+active+clean, 19 active+undersized, 118 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 54/726 objects degraded (7.438%) 2024-06-01T02:09:33.984 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[89502]: from='client.64390 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:09:33.984 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[89502]: Health check failed: Degraded data redundancy: 54/726 objects degraded (7.438%), 14 pgs degraded (PG_DEGRADED) 2024-06-01T02:09:33.984 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.15", "id": [3, 2]}]': finished 2024-06-01T02:09:33.984 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.16", "id": [7, 4]}]': finished 2024-06-01T02:09:33.984 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [0, 4]}]': finished 2024-06-01T02:09:33.984 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.0", "id": [3, 4]}]': finished 2024-06-01T02:09:33.984 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2, 3, 4]}]': finished 2024-06-01T02:09:33.984 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]': finished 2024-06-01T02:09:33.984 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.3"}]': finished 2024-06-01T02:09:33.985 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.e"}]': finished 2024-06-01T02:09:33.985 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.14"}]': finished 2024-06-01T02:09:33.985 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.1c"}]': finished 2024-06-01T02:09:33.985 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[89502]: osdmap e156: 8 total, 7 up, 8 in 2024-06-01T02:09:33.985 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[89502]: from='client.44430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:09:33.985 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/581318795' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:33.985 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:33.985 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:33.985 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:33.985 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:33.985 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:33.985 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:09:33.986 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[100606]: pgmap v199: 161 pgs: 14 active+undersized+degraded, 10 stale+active+clean, 19 active+undersized, 118 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 54/726 objects degraded (7.438%) 2024-06-01T02:09:33.986 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[100606]: from='client.64390 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:09:33.986 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[100606]: Health check failed: Degraded data redundancy: 54/726 objects degraded (7.438%), 14 pgs degraded (PG_DEGRADED) 2024-06-01T02:09:33.986 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.15", "id": [3, 2]}]': finished 2024-06-01T02:09:33.986 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.16", "id": [7, 4]}]': finished 2024-06-01T02:09:33.986 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [0, 4]}]': finished 2024-06-01T02:09:33.986 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.0", "id": [3, 4]}]': finished 2024-06-01T02:09:33.986 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2, 3, 4]}]': finished 2024-06-01T02:09:33.986 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]': finished 2024-06-01T02:09:33.986 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.3"}]': finished 2024-06-01T02:09:33.986 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.e"}]': finished 2024-06-01T02:09:33.986 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.14"}]': finished 2024-06-01T02:09:33.986 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.1c"}]': finished 2024-06-01T02:09:33.987 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[100606]: osdmap e156: 8 total, 7 up, 8 in 2024-06-01T02:09:33.987 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[100606]: from='client.44430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:09:33.987 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/581318795' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:33.987 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:33.987 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:33.987 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:33.987 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:33.987 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:34.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:33 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:09:34.089 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:33 smithi161 ceph-mon[84406]: pgmap v199: 161 pgs: 14 active+undersized+degraded, 10 stale+active+clean, 19 active+undersized, 118 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 54/726 objects degraded (7.438%) 2024-06-01T02:09:34.089 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:33 smithi161 ceph-mon[84406]: from='client.64390 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:09:34.089 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:33 smithi161 ceph-mon[84406]: Health check failed: Degraded data redundancy: 54/726 objects degraded (7.438%), 14 pgs degraded (PG_DEGRADED) 2024-06-01T02:09:34.089 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.15", "id": [3, 2]}]': finished 2024-06-01T02:09:34.089 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.16", "id": [7, 4]}]': finished 2024-06-01T02:09:34.089 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [0, 4]}]': finished 2024-06-01T02:09:34.089 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.0", "id": [3, 4]}]': finished 2024-06-01T02:09:34.089 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2, 3, 4]}]': finished 2024-06-01T02:09:34.089 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]': finished 2024-06-01T02:09:34.089 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.3"}]': finished 2024-06-01T02:09:34.089 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.e"}]': finished 2024-06-01T02:09:34.089 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.14"}]': finished 2024-06-01T02:09:34.090 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.1c"}]': finished 2024-06-01T02:09:34.090 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:33 smithi161 ceph-mon[84406]: osdmap e156: 8 total, 7 up, 8 in 2024-06-01T02:09:34.090 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:33 smithi161 ceph-mon[84406]: from='client.44430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:09:34.090 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:33 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/581318795' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:34.090 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:34.090 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:34.090 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:34.090 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:34.090 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:34.387 INFO:teuthology.orchestra.run.smithi116.stdout:HEALTH_WARN 1 osds down; Degraded data redundancy: 54/726 objects degraded (7.438%), 14 pgs degraded 2024-06-01T02:09:34.388 INFO:teuthology.orchestra.run.smithi116.stdout:[WRN] OSD_DOWN: 1 osds down 2024-06-01T02:09:34.388 INFO:teuthology.orchestra.run.smithi116.stdout: osd.4 (root=default,host=smithi161) is down 2024-06-01T02:09:34.388 INFO:teuthology.orchestra.run.smithi116.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 54/726 objects degraded (7.438%), 14 pgs degraded 2024-06-01T02:09:34.388 INFO:teuthology.orchestra.run.smithi116.stdout: pg 2.b is active+undersized+degraded, acting [7,5] 2024-06-01T02:09:34.388 INFO:teuthology.orchestra.run.smithi116.stdout: pg 2.11 is active+undersized+degraded, acting [6,1] 2024-06-01T02:09:34.388 INFO:teuthology.orchestra.run.smithi116.stdout: pg 2.18 is active+undersized+degraded, acting [5,1] 2024-06-01T02:09:34.388 INFO:teuthology.orchestra.run.smithi116.stdout: pg 3.9 is active+undersized+degraded, acting [2,7] 2024-06-01T02:09:34.388 INFO:teuthology.orchestra.run.smithi116.stdout: pg 3.a is active+undersized+degraded, acting [6,1] 2024-06-01T02:09:34.388 INFO:teuthology.orchestra.run.smithi116.stdout: pg 3.f is active+undersized+degraded, acting [7,0] 2024-06-01T02:09:34.388 INFO:teuthology.orchestra.run.smithi116.stdout: pg 3.11 is active+undersized+degraded, acting [7,6] 2024-06-01T02:09:34.388 INFO:teuthology.orchestra.run.smithi116.stdout: pg 3.13 is active+undersized+degraded, acting [7,2] 2024-06-01T02:09:34.388 INFO:teuthology.orchestra.run.smithi116.stdout: pg 3.15 is active+undersized+degraded, acting [7,3] 2024-06-01T02:09:34.389 INFO:teuthology.orchestra.run.smithi116.stdout: pg 3.1d is active+undersized+degraded, acting [5,6] 2024-06-01T02:09:34.389 INFO:teuthology.orchestra.run.smithi116.stdout: pg 4.d is active+undersized+degraded, acting [2,1] 2024-06-01T02:09:34.389 INFO:teuthology.orchestra.run.smithi116.stdout: pg 4.e is active+undersized+degraded, acting [6,1] 2024-06-01T02:09:34.389 INFO:teuthology.orchestra.run.smithi116.stdout: pg 5.1d is active+undersized+degraded, acting [6,0] 2024-06-01T02:09:34.389 INFO:teuthology.orchestra.run.smithi116.stdout: pg 6.1a is active+undersized+degraded, acting [5,1] 2024-06-01T02:09:34.444 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[101715]: 2024-06-01T02:09:34.092+0000 7f004dc327c0 -1 Falling back to public interface 2024-06-01T02:09:35.186 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='client.44436 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:09:35.186 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:35.186 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: osdmap e157: 8 total, 7 up, 8 in 2024-06-01T02:09:35.186 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:35.186 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:35.187 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:35.187 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:35.187 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:35.187 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:35.187 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:35.187 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:35.187 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:35.187 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:35.187 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:35.187 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:35.187 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:35.187 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:35.187 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:35.187 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:35.187 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:35.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:35.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:35.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:35.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:35.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/2272361582' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:09:35.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:35.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:35.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:35.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:35.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:35.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:35.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:35.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:35.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:35.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:35.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:35.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:35.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:35.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:35.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:35.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='client.44436 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:09:35.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:35.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: osdmap e157: 8 total, 7 up, 8 in 2024-06-01T02:09:35.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:35.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:35.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:35.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:35.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:35.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:35.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:35.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:35.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:35.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:35.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:35.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:35.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:35.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:35.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:35.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:35.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:35.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:35.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:35.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:35.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:35.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/2272361582' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:09:35.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:35.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:35.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:35.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:35.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:35.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:35.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:35.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:35.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:35.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:35.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:35.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:35.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:35.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:35.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:35.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='client.44436 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:09:35.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:35.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: osdmap e157: 8 total, 7 up, 8 in 2024-06-01T02:09:35.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:35.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:35.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:35.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:35.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:35.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:35.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:35.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:35.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:35.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:35.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:35.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:35.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:35.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:35.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:35.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:35.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:35.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:35.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:35.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:35.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:35.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/2272361582' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:09:35.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:35.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:35.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:35.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:35.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:35.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:35.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:35.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:35.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:35.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:35.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:35.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:35.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:35.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:35.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:35.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='client.44451 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:09:35.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:35.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:35.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: pgmap v202: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 908 B/s rd, 0 op/s; 98/726 objects degraded (13.499%) 2024-06-01T02:09:35.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:35.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:35.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:35.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:35.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: osdmap e158: 8 total, 7 up, 8 in 2024-06-01T02:09:35.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:35.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:35.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:35.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:35.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:35.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:35.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:35.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:35.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:35.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:35.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:35.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:35.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:35.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:35.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:35.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:35.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:35.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:35.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:35.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:35.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:35.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:35.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:35.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:35.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:35.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:35.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:35.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:35.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:35.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:35.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:35.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:35.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:35.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:35.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:35.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:35.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:35.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:35.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:35.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:36.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='client.44451 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:09:36.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:36.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:36.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: pgmap v202: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 908 B/s rd, 0 op/s; 98/726 objects degraded (13.499%) 2024-06-01T02:09:36.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:36.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:36.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:36.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:36.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: osdmap e158: 8 total, 7 up, 8 in 2024-06-01T02:09:36.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:36.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:36.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:36.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:36.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:36.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:36.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:36.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:36.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:36.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:36.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:36.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:36.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:36.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:36.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:36.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:36.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:36.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:36.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:36.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:36.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='client.44451 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:09:36.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:36.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:36.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: pgmap v202: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 908 B/s rd, 0 op/s; 98/726 objects degraded (13.499%) 2024-06-01T02:09:36.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:36.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:36.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:36.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:36.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: osdmap e158: 8 total, 7 up, 8 in 2024-06-01T02:09:36.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:36.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:36.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:36.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:36.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:36.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:36.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:36.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:36.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:36.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:36.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:36.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:36.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:36.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:36.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:36.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:36.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:36.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:36.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:36.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:36.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:36.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:36.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:36.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:36.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:36.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:36.302 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:36.302 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:36.302 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:36.302 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:36.302 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:36.302 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:36.302 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:36.302 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:36.302 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:36.303 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:36.303 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:36.303 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:36.303 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:36.303 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:36.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:36.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:36.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:36.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:36.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:36.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:36.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:36.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:36.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:36.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:36.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:36.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:36.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:36.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:36.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:36.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:36.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:36.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:36.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:36.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:37.082 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:09:36] "GET /metrics HTTP/1.1" 200 38456 "" "Prometheus/2.33.4" 2024-06-01T02:09:37.083 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[101715]: 2024-06-01T02:09:36.862+0000 7f004dc327c0 -1 osd.4 153 log_to_monitors true 2024-06-01T02:09:37.083 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:37.083 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:37.083 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:37.083 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:37.084 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:37.084 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:37.084 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:37.084 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:37.084 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:37.084 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:37.084 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:37.084 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:37.084 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:37.084 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:37.084 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:37.084 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:37.084 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:37.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:37.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:37.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:37.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:37.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:37.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:37.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:37.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:37.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:37.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:37.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:37.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:37.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:37.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:37.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:37.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:37.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:37.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:37.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:37.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:37.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:37.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:37.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:37.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:37.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:37.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:37.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:37.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:37.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:37.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:37.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:37.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:37.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:36 smithi161 ceph-mon[84406]: from='osd.4 [v2:172.21.15.161:6800/3486552979,v1:172.21.15.161:6801/3486552979]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-01T02:09:37.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:37.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:37.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:37.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:37.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:37.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:37.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:37.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:37.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:37.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:37.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:37.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:37.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:37.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:37.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:37.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:37.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:37.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:37.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:37.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:37.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:37.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:37.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:37.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:37.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:37.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:37.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:37.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:37.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:37.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:37.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:37.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:37.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:37.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:37.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:37.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:37.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:37.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:37.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:37.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:37.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:37.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:37.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:37.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:37.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:37.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:37.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:37.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:37.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:37.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:37.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:37.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:37.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:37.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:37.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:37.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:37.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:37.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:37.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:37.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:37.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:37.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:37.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:37.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:37.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[100606]: from='osd.4 [v2:172.21.15.161:6800/3486552979,v1:172.21.15.161:6801/3486552979]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-01T02:09:37.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:37.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:37.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:37.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:37.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:37.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:37.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:37.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:37.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:37.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:37.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:37.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:37.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:37.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:37.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:37.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:37.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:37.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:37.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:37.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:36 smithi116 ceph-mon[89502]: from='osd.4 [v2:172.21.15.161:6800/3486552979,v1:172.21.15.161:6801/3486552979]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-01T02:09:38.067 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[101715]: 2024-06-01T02:09:37.826+0000 7f003b081700 -1 osd.4 153 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-01T02:09:38.068 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:38.068 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:38.068 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: pgmap v204: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 98/726 objects degraded (13.499%) 2024-06-01T02:09:38.068 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:38.068 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:38.068 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:38.068 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:38.068 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:38.068 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-01T02:09:38.068 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:38.069 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:38.069 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:38.069 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:38.069 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:38.069 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:38.069 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:38.069 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:38.069 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:38.069 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:38.069 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:38.069 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:38.069 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:38.069 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:38.070 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:38.070 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:38.070 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:38.070 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:38.070 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:38.070 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:38.070 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:38.070 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:38.070 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:38.070 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:38.070 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:38.070 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:38.070 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:38.071 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:38.071 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:38.071 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:38.071 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:38.071 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:38.071 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:38.071 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:38.071 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:38.071 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:38.071 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-01T02:09:38.071 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: osdmap e159: 8 total, 7 up, 8 in 2024-06-01T02:09:38.071 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='osd.4 [v2:172.21.15.161:6800/3486552979,v1:172.21.15.161:6801/3486552979]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:09:38.071 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:37 smithi161 ceph-mon[84406]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:09:38.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:38.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:38.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: pgmap v204: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 98/726 objects degraded (13.499%) 2024-06-01T02:09:38.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:38.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:38.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:38.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:38.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:38.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-01T02:09:38.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:38.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:38.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:38.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:38.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:38.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:38.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:38.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:38.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:38.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:38.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:38.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:38.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:38.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:38.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:38.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:38.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:38.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:38.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:38.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:38.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:38.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: pgmap v204: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 98/726 objects degraded (13.499%) 2024-06-01T02:09:38.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:38.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:38.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:38.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:38.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:38.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-01T02:09:38.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:38.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:38.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:38.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:38.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:38.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:38.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:38.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:38.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:38.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:38.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:38.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:38.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:38.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:38.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:38.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:38.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:38.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:38.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:38.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:38.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:38.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:38.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:38.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:38.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:38.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:38.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:38.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:38.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:38.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:38.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:38.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:38.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:38.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:38.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:38.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:38.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:38.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-01T02:09:38.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: osdmap e159: 8 total, 7 up, 8 in 2024-06-01T02:09:38.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='osd.4 [v2:172.21.15.161:6800/3486552979,v1:172.21.15.161:6801/3486552979]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:09:38.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[100606]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:09:38.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:38.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:38.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:38.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:38.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:38.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:38.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:38.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:38.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:38.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:38.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:38.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:38.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:38.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:38.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:38.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:38.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-01T02:09:38.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: osdmap e159: 8 total, 7 up, 8 in 2024-06-01T02:09:38.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='osd.4 [v2:172.21.15.161:6800/3486552979,v1:172.21.15.161:6801/3486552979]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:09:38.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:37 smithi116 ceph-mon[89502]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:09:39.145 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:39.145 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:39.145 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:39.146 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:39.146 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:39.146 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:39.146 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:39.146 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:39.146 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:39.146 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:39.146 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:39.146 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:39.146 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:39.146 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:39.146 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:39.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:39.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:39.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:39.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:39.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:39.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:39.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:39.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:39.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:39.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:39.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:39.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:39.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:39.147 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:39.148 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:39.148 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:39.148 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:39.148 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:39.148 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:39.148 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:39.148 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:39.148 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:39.148 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:39.148 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:39.148 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:39.148 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:39.148 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:39.148 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:39.149 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:39.149 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:39.149 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:39.149 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:39.149 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:09:39.149 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: osd.4 [v2:172.21.15.161:6800/3486552979,v1:172.21.15.161:6801/3486552979] boot 2024-06-01T02:09:39.149 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: osdmap e160: 8 total, 8 up, 8 in 2024-06-01T02:09:39.149 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T02:09:39.149 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:39.149 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:39.149 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:39.149 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:39.149 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:39.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:39.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:39.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:39.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:39.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:39.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:39.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:39.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:39.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:39.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:39.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:39.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:39.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:39.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:39.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:39.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:39.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:39.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:39.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:39.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:39.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:39.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:39.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:39.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:39.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:39.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:39.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:39.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:39.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:39.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:39.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:39.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:39.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:39.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:39.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:39.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:39.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:39.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:39.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:39.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:39.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:39.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:39.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:39.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:39.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:39.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:39.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:39.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:39.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:39.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:39.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:39.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:39.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:39.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:39.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:39.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:39.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:39.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:39.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:39.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:39.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:39.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:39.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:39.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:39.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:39.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:39.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:39.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:39.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:39.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:39.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:39.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:39.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:39.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:39.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:39.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:39.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:09:39.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: osd.4 [v2:172.21.15.161:6800/3486552979,v1:172.21.15.161:6801/3486552979] boot 2024-06-01T02:09:39.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: osdmap e160: 8 total, 8 up, 8 in 2024-06-01T02:09:39.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T02:09:39.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:39.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:39.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:39.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:39.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:39.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:39.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:39.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:39.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:39.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:39.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:39.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:39.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:39.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:39.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:39.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:39.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:39.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:39.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:39.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:39.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:39.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:39.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:39.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:09:39.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: osd.4 [v2:172.21.15.161:6800/3486552979,v1:172.21.15.161:6801/3486552979] boot 2024-06-01T02:09:39.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: osdmap e160: 8 total, 8 up, 8 in 2024-06-01T02:09:39.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-01T02:09:39.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:39.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:39.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:39.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:39.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:40.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:40.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: pgmap v206: 161 pgs: 9 active+clean+remapped, 23 active+undersized+degraded, 39 active+undersized, 90 active+clean; 585 KiB data, 208 MiB used, 715 GiB / 715 GiB avail; 382 B/s rd, 0 op/s; 93/726 objects degraded (12.810%); 47/726 objects misplaced (6.474%); 20 B/s, 0 objects/s recovering 2024-06-01T02:09:40.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:40.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:40.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:40.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:40.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:40.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:40.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:40.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:40.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:40.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:40.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:40.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:40.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:40.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:40.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:40.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:40.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:40.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:40.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: Health check update: Degraded data redundancy: 93/726 objects degraded (12.810%), 23 pgs degraded (PG_DEGRADED) 2024-06-01T02:09:40.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:40.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:40.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:40.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:40.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:40.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:40.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:40.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:40.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:40.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:40.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:40.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:40.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:40.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:40.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:40.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:40.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:40.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:40.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:40.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:40.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:40.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:40.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:40.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:40.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:40.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:40.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:40.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:40.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:40.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:40.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:39 smithi161 ceph-mon[84406]: osdmap e161: 8 total, 8 up, 8 in 2024-06-01T02:09:40.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:40.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: pgmap v206: 161 pgs: 9 active+clean+remapped, 23 active+undersized+degraded, 39 active+undersized, 90 active+clean; 585 KiB data, 208 MiB used, 715 GiB / 715 GiB avail; 382 B/s rd, 0 op/s; 93/726 objects degraded (12.810%); 47/726 objects misplaced (6.474%); 20 B/s, 0 objects/s recovering 2024-06-01T02:09:40.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:40.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:40.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:40.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:40.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:40.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:40.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:40.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:40.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:40.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:40.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:40.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:40.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:40.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:40.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:40.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:40.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:40.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:40.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: Health check update: Degraded data redundancy: 93/726 objects degraded (12.810%), 23 pgs degraded (PG_DEGRADED) 2024-06-01T02:09:40.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:40.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:40.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:40.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:40.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:40.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:40.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:40.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:40.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: pgmap v206: 161 pgs: 9 active+clean+remapped, 23 active+undersized+degraded, 39 active+undersized, 90 active+clean; 585 KiB data, 208 MiB used, 715 GiB / 715 GiB avail; 382 B/s rd, 0 op/s; 93/726 objects degraded (12.810%); 47/726 objects misplaced (6.474%); 20 B/s, 0 objects/s recovering 2024-06-01T02:09:40.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:40.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:40.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:40.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:40.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:40.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:40.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:40.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:40.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:40.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:40.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:40.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:40.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:40.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:40.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:40.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:40.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:40.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:40.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: Health check update: Degraded data redundancy: 93/726 objects degraded (12.810%), 23 pgs degraded (PG_DEGRADED) 2024-06-01T02:09:40.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:40.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:40.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:40.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:40.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:40.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:40.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:40.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:40.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:40.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:40.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:40.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:40.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:40.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:40.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:40.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:40.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:40.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:40.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:40.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:40.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:40.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:40.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:40.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:40.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:40.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:40.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:40.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:40.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:40.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:40.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[100606]: osdmap e161: 8 total, 8 up, 8 in 2024-06-01T02:09:40.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:40.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:40.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:40.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:40.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:40.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:40.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:40.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:40.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:40.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:40.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:40.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:40.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:40.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:40.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:40.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:40.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:40.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:40.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:40.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:40.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:40.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:40.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:40.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:39 smithi116 ceph-mon[89502]: osdmap e161: 8 total, 8 up, 8 in 2024-06-01T02:09:41.163 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:41.163 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:41.164 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:41.164 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:41.164 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:41.164 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:41.164 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:41.164 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:41.164 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:41.164 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:41.164 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:41.164 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:41.164 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:41.164 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:41.164 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:41.165 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:41.165 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:41.165 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:41.165 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:41.165 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:41.165 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:41.165 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:41.165 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:41.165 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:41.165 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:41.165 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:41.165 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:41.165 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:41.165 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:41.166 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:41.166 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:41.166 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:41.166 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:41.166 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:41.166 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:41.166 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:41.166 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:41.166 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:41.166 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:41.166 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:41.166 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:41.166 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:41.167 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:41.167 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: osdmap e162: 8 total, 8 up, 8 in 2024-06-01T02:09:41.167 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:41.167 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:41.167 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:41.167 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:41.167 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:41.167 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:41.167 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:41.167 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:41.168 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:41.168 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:41.168 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:41.168 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:41.168 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:41.168 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:41.168 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:41.168 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:41.168 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:41.168 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:41.168 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:41.168 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:41.168 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:41.168 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:41.169 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:41.169 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:41.169 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:41.169 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:41.169 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:41.169 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:41.169 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:41.169 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:41.169 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:41.169 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:41.169 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:41.169 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:41.170 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:41.170 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:41.170 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:41.170 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:41.170 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:41.170 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:41.170 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:41.170 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:41.170 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:41.170 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:41.170 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:41.170 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:41.171 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:41.171 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:41.171 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: osdmap e162: 8 total, 8 up, 8 in 2024-06-01T02:09:41.171 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:41.171 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:41.171 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:41.171 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:41.171 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:41.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:41.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:41.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:41.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:41.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:41.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:41.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:41.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:41.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:41.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:41.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:41.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:41.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:41.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:41.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:41.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:41.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:41.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:41.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:41.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:41.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:41.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:41.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:41.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:41.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:41.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:41.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:41.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:41.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:41.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:41.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:41.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:41.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:41.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:41.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:41.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:41.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:41.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:41.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:41.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:41.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:41.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:41.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:41.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: osdmap e162: 8 total, 8 up, 8 in 2024-06-01T02:09:41.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:41.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:41.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:41.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:41.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:42.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:42.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:42.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: pgmap v209: 161 pgs: 9 active+clean+remapped, 23 active+undersized+degraded, 39 active+undersized, 90 active+clean; 585 KiB data, 208 MiB used, 715 GiB / 715 GiB avail; 382 B/s rd, 0 op/s; 93/726 objects degraded (12.810%); 47/726 objects misplaced (6.474%); 20 B/s, 0 objects/s recovering 2024-06-01T02:09:42.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:42.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:42.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:42.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:42.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:42.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:42.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:42.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:42.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:42.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:42.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:42.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:42.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:42.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:42.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:42.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:42.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:42.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:42.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:42.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:42.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: osdmap e163: 8 total, 8 up, 8 in 2024-06-01T02:09:42.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:42.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:42.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:42.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:42.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:42.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:42.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:42.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:42.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:42.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:42.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:42.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:42.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:42.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:42.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:42.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:42.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:42.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:42.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:42.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:42.220 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:42.220 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:42.220 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: pgmap v209: 161 pgs: 9 active+clean+remapped, 23 active+undersized+degraded, 39 active+undersized, 90 active+clean; 585 KiB data, 208 MiB used, 715 GiB / 715 GiB avail; 382 B/s rd, 0 op/s; 93/726 objects degraded (12.810%); 47/726 objects misplaced (6.474%); 20 B/s, 0 objects/s recovering 2024-06-01T02:09:42.220 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:42.220 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:42.220 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:42.220 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:42.220 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:42.220 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:42.220 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:42.220 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:42.220 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:42.220 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:42.221 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:42.221 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:42.221 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:42.221 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:42.221 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:42.221 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:42.221 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:42.221 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:42.221 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:42.221 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:42.221 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: osdmap e163: 8 total, 8 up, 8 in 2024-06-01T02:09:42.221 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:42.221 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:42.221 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:42.222 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:42.222 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:42.222 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:42.222 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:42.222 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:42.222 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:42.222 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:42.222 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:42.222 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:42.222 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:42.222 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:42.222 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:42.222 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:42.222 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:42.223 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:42.223 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:42.223 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:42.223 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:42.223 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:42.223 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: pgmap v209: 161 pgs: 9 active+clean+remapped, 23 active+undersized+degraded, 39 active+undersized, 90 active+clean; 585 KiB data, 208 MiB used, 715 GiB / 715 GiB avail; 382 B/s rd, 0 op/s; 93/726 objects degraded (12.810%); 47/726 objects misplaced (6.474%); 20 B/s, 0 objects/s recovering 2024-06-01T02:09:42.223 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:42.223 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:42.223 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:42.223 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:42.224 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:42.224 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:42.224 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:42.224 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:42.224 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:42.224 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:42.224 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:42.224 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:42.224 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:42.224 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:42.224 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:42.224 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:42.224 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:42.224 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:42.225 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:42.225 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:42.225 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: osdmap e163: 8 total, 8 up, 8 in 2024-06-01T02:09:42.225 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:42.225 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:42.225 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:42.225 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:42.225 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:42.225 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:42.225 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:42.225 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:42.225 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:42.225 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:42.225 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:42.226 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:42.226 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:42.226 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:42.226 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:42.226 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:42.226 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:42.226 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:42.226 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:42.226 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:42.881 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:42.881 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:42.882 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:42.882 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:42.882 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:42.882 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:42.882 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:42.882 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:42.883 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:42.883 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:42.883 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:42.883 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:42.883 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:42.883 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:42.883 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:42.883 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:42.883 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:42.883 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:42.883 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:42.883 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:42.883 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:42.884 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:42.884 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:42.884 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:42.884 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:42.884 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:42.884 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:42.884 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: osdmap e164: 8 total, 8 up, 8 in 2024-06-01T02:09:42.884 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:42.884 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:42.884 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:42.884 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:42.884 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:42.884 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:42.884 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:42.885 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:42.885 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:42.885 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:42.885 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:42.885 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:42.885 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:42.885 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:42.885 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:42.885 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:43.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:43.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:43.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:43.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:43.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:43.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:43.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:43.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:43.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:43.231 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:43.232 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:43.232 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:43.232 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:43.232 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:43.232 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:43.232 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:43.232 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:43.232 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:43.232 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:43.232 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:43.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:43.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:43.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:43.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:43.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:43.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:43.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:43.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:43.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:43.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:43.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:43.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:43.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:43.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:43.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:43.234 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:43.234 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: osdmap e164: 8 total, 8 up, 8 in 2024-06-01T02:09:43.234 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:43.234 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:43.234 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:43.234 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:43.234 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:43.234 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:43.234 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:43.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:43.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:43.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:43.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:43.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:43.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:43.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:43.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:43.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:43.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:43.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:43.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:43.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:43.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:43.235 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:43.236 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:43.236 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:43.236 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:43.236 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:43.236 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:43.236 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:43.236 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:43.236 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: osdmap e164: 8 total, 8 up, 8 in 2024-06-01T02:09:43.236 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:43.236 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:43.237 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:43.237 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:43.237 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:43.237 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:43.237 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:43.237 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:43.237 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:43.237 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:43.237 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:43.237 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:43.237 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:43.237 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:43.238 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:43.238 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:43.238 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:43.238 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:43.238 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:43.238 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:43.238 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:43.238 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:43.238 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:43.238 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:43.238 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:43.239 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:43.239 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:43.239 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:43.239 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:43.239 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:43.239 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:43.239 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:43.239 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:43.239 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:43.239 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:43.239 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:43.239 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:43.239 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:43.239 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:43.239 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:43.240 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:43.240 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:43.240 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:43.240 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:43.240 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:43.240 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:43.240 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:43.240 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:44.154 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:44.154 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:09:44.154 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:44.154 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:44.154 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: pgmap v212: 161 pgs: 3 active+undersized+remapped, 4 active+clean+remapped, 14 active+undersized+degraded, 27 active+undersized, 113 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 54/726 objects degraded (7.438%); 15/726 objects misplaced (2.066%); 82 B/s, 6 objects/s recovering 2024-06-01T02:09:44.154 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:44.154 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:44.154 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:44.155 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:44.155 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:44.155 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:44.155 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:44.155 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:44.155 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:44.155 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:44.155 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:44.155 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:44.155 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:44.155 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:44.155 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:44.155 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:44.156 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:44.156 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:44.156 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:44.156 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:44.156 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:44.156 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: osdmap e165: 8 total, 8 up, 8 in 2024-06-01T02:09:44.156 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:44.156 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:44.157 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:44.157 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:44.157 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:44.157 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:44.157 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:44.157 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:44.157 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:44.157 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:44.157 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:44.157 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:44.157 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:44.158 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:44.158 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:44.158 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:44.158 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:44.158 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:44.158 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:44.158 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:44.158 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:44.158 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:44.158 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:44.158 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:44.158 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:44.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:44.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:09:44.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:44.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:44.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: pgmap v212: 161 pgs: 3 active+undersized+remapped, 4 active+clean+remapped, 14 active+undersized+degraded, 27 active+undersized, 113 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 54/726 objects degraded (7.438%); 15/726 objects misplaced (2.066%); 82 B/s, 6 objects/s recovering 2024-06-01T02:09:44.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:44.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:44.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:44.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:44.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:44.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:44.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:44.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:44.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:44.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:44.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:44.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:44.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:44.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:44.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:44.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:44.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:44.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:44.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:44.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:44.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:44.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: osdmap e165: 8 total, 8 up, 8 in 2024-06-01T02:09:44.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:44.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:44.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:44.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:09:44.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:44.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:44.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: pgmap v212: 161 pgs: 3 active+undersized+remapped, 4 active+clean+remapped, 14 active+undersized+degraded, 27 active+undersized, 113 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 54/726 objects degraded (7.438%); 15/726 objects misplaced (2.066%); 82 B/s, 6 objects/s recovering 2024-06-01T02:09:44.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:44.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:44.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:44.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:44.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:44.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:44.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:44.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:44.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:44.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:44.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:44.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:44.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:44.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:44.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:44.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:44.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:44.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:44.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:44.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:44.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:44.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: osdmap e165: 8 total, 8 up, 8 in 2024-06-01T02:09:44.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:44.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:44.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:44.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:44.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:44.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:44.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:44.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:44.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:44.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:44.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:44.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:44.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:44.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:44.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:44.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:44.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:44.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:44.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:44.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:44.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:44.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:44.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:44.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:44.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:44.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:44.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:44.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:44.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:44.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:44.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:44.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:44.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:44.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:44.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:44.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:44.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:44.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:44.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:44.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:44.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:44.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:44.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:44.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:44.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:44.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:44.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:44.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:45.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:45.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:45.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:45.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:45.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:45.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:45.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:45.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:45.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:45.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:45.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:45.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:45.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:45.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:45.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:45.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:45.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:45.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:45.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:45.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:45.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:45.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:45.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:45.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:45.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:45.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:45.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:45.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:45.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:45.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:45.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:45.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:45.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:45.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:45.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:45.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:45.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:45.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:45.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:45.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:45.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:45.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:45.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:45.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:45.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:45.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:45.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:45.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:45.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:45.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:45.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:45.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:45.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:45.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:45.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:45.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:45.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:45.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:45.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:45.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:45.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:45.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:45.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:45.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:45.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:45.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:45.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:45.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:45.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:45.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:45.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:45.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:45.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:45.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:45.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:45.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:45.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:45.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:45.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:45.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:45.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:45.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:45.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:45.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:45.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:45.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:45.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:45.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:45.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:45.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:45.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:45.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:45.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:45.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:45.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:45.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:45.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:45.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:45.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:45.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:45.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:45.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:45.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:45.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:45.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:45.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:45.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:45.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:45.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:45.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:45.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:45.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:45.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:45.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:45.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:45.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:45.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:45.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:45.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:45.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:45.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:45.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:45.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:45.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:45.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:45.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:45.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:45.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:45.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:45.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:45.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:45.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:45.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:45.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:45.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:45.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:45.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:45.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:45.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:45.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:45.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:45.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:45.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:45.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:45.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:45.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:45.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:45.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:45.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:45.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:45.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:45.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:45.302 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:45.302 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:45.302 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:45.302 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:45.302 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:45.302 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:45.302 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:46.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:46.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:46.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: pgmap v215: 161 pgs: 2 active+undersized+remapped, 1 peering, 158 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 164 B/s, 8 objects/s recovering 2024-06-01T02:09:46.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:46.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:46.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:46.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:46.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:46.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:46.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:46.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:46.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:46.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:46.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:46.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:46.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:46.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:46.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:46.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:46.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-01T02:09:46.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 54/726 objects degraded (7.438%), 14 pgs degraded) 2024-06-01T02:09:46.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:46.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:46.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:46.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:46.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:46.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:46.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:46.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:46.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:46.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:46.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:46.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:46.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:46.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:46.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:46.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:46.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:46.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:46.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:46.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:46.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:46.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:46.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:46.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:46.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:46.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:46.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:46.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:46.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:46.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:46.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:46.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:46.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:09:46.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:46.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:46.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: pgmap v215: 161 pgs: 2 active+undersized+remapped, 1 peering, 158 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 164 B/s, 8 objects/s recovering 2024-06-01T02:09:46.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:46.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:46.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:46.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:46.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:46.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:46.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:46.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:46.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:46.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:46.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:46.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:46.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:46.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:46.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:46.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:46.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-01T02:09:46.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 54/726 objects degraded (7.438%), 14 pgs degraded) 2024-06-01T02:09:46.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:46.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:46.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:46.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:46.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:46.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:46.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:46.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:46.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:46.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:46.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: pgmap v215: 161 pgs: 2 active+undersized+remapped, 1 peering, 158 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 164 B/s, 8 objects/s recovering 2024-06-01T02:09:46.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:46.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:46.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:46.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:46.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:46.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:46.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:46.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:46.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:46.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:46.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:46.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:46.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:46.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:46.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:46.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:46.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-01T02:09:46.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 54/726 objects degraded (7.438%), 14 pgs degraded) 2024-06-01T02:09:46.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:46.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:46.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:46.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:46.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:46.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:46.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:46.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:46.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:46.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:46.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:46.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:46.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:46.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:46.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:46.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:46.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:46.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:46.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:46.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:46.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:46.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:46.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:46.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:46.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:46.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:46.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:46.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:46.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:46.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:46.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:46.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:46.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:09:46.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:46.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:46.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:46.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:46.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:46.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:46.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:46.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:46.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:46.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:46.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:46.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:46.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:46.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:46.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:46.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:46.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:46.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:46.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:46.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:46.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:46.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:46.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:46.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:46.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:09:46.890 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:09:46] "GET /metrics HTTP/1.1" 200 38527 "" "Prometheus/2.33.4" 2024-06-01T02:09:47.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:47.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:47.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:47.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:47.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:47.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:47.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:47.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:47.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:47.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:47.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:47.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:47.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:47.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:47.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:47.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:47.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:47.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:47.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:47.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:47.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:47.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:47.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:47.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:47.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:47.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:47.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:47.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:47.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:47.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:47.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:47.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:47.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:47.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:47.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:47.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:47.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:47.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:47.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:47.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:47.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:47.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:47.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:47.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:47.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:47.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:47.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:47.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:47.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:47.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:47.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:47.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:47.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:47.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:47.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:47.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:47.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:47.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:47.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:47.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:47.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:47.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:47.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:47.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:47.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:47.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:47.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:47.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:47.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:47.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:47.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:47.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:47.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:47.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:47.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:47.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:47.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:47.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:47.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:47.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:47.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:47.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:47.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:47.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:47.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:47.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:47.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:47.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:47.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:47.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:47.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:47.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:47.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:47.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:47.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:47.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:47.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:47.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:47.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:47.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:47.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:47.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:47.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:47.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:47.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:47.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:47.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:47.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:47.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:47.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:47.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:47.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:47.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:47.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:47.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:47.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:47.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:47.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:47.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:47.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:47.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:47.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:47.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:47.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:47.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:47.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:47.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:47.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:47.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:47.694 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:09:47.614+0000 7fcedaabb700 -1 mgr.server reply reply (16) Device or resource busy unsafe to stop osd(s) at this time (2 PGs are or would become offline) 2024-06-01T02:09:48.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:48.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:48.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: pgmap v216: 161 pgs: 2 active+undersized+remapped, 1 peering, 158 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 121 B/s, 6 objects/s recovering 2024-06-01T02:09:48.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:48.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:48.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:48.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:48.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:48.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:48.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:48.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:48.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:48.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:48.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:48.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:48.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:48.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:48.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:48.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:48.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:48.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:48.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:48.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:48.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:48.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:48.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:48.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:48.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:48.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:48.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:48.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:48.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:48.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:48.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:48.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:48.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:48.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:48.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:48.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:48.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:48.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-01T02:09:48.195 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:48.195 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:48.195 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: pgmap v216: 161 pgs: 2 active+undersized+remapped, 1 peering, 158 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 121 B/s, 6 objects/s recovering 2024-06-01T02:09:48.195 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:48.195 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:48.195 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:48.195 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:48.195 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:48.196 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:48.196 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:48.196 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:48.196 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:48.196 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:48.196 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:48.196 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:48.196 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:48.197 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:48.197 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:48.197 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:48.197 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:48.197 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:48.197 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:48.197 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:48.197 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:48.197 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:48.197 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:48.197 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:48.197 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:48.198 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:48.198 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:48.198 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: pgmap v216: 161 pgs: 2 active+undersized+remapped, 1 peering, 158 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 121 B/s, 6 objects/s recovering 2024-06-01T02:09:48.198 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:48.198 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:48.198 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:48.198 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:48.198 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:48.198 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:48.198 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:48.198 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:48.198 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:48.198 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:48.199 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:48.199 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:48.199 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:48.199 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:48.199 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:48.199 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:48.199 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:48.199 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:48.200 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:48.200 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:48.200 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:48.200 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:48.200 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:48.200 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:48.200 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:48.201 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:48.202 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:48.202 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:48.202 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:48.202 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:48.202 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:48.202 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:48.203 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:48.203 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:48.203 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:48.203 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:48.203 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:48.203 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:48.204 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-01T02:09:48.204 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:48.204 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:09:48.204 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:09:48.204 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:48.205 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:48.205 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:48.205 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:48.205 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:09:48.205 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:48.205 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:48.206 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:09:48.206 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-01T02:09:49.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:48 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:49.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:48 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:49.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:48 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:49.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:48 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:49.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:48 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:49.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:48 smithi161 ceph-mon[84406]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-01T02:09:49.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:48 smithi161 ceph-mon[84406]: Upgrade: unsafe to stop osd(s) at this time (2 PGs are or would become offline) 2024-06-01T02:09:49.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:49.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:49.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:49.207 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:48 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:49.207 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:48 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:49.207 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:48 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:49.207 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:48 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:49.207 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:48 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:49.208 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:48 smithi116 ceph-mon[89502]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-01T02:09:49.208 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:48 smithi116 ceph-mon[89502]: Upgrade: unsafe to stop osd(s) at this time (2 PGs are or would become offline) 2024-06-01T02:09:49.208 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:49.208 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:49.208 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:49.208 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:48 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:49.208 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:48 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:49.208 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:48 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:49.208 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:48 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:49.208 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:48 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:09:49.208 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:48 smithi116 ceph-mon[100606]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-01T02:09:49.208 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:48 smithi116 ceph-mon[100606]: Upgrade: unsafe to stop osd(s) at this time (2 PGs are or would become offline) 2024-06-01T02:09:49.209 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:49.209 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:49.209 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:50.158 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:49 smithi161 ceph-mon[84406]: pgmap v217: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s; 95 B/s, 5 objects/s recovering 2024-06-01T02:09:50.158 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:49 smithi161 ceph-mon[84406]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-01T02:09:50.158 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:49 smithi161 ceph-mon[84406]: Cluster is now healthy 2024-06-01T02:09:50.158 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:50.158 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:50.158 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:50.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:49 smithi116 ceph-mon[100606]: pgmap v217: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s; 95 B/s, 5 objects/s recovering 2024-06-01T02:09:50.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:49 smithi116 ceph-mon[100606]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-01T02:09:50.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:49 smithi116 ceph-mon[100606]: Cluster is now healthy 2024-06-01T02:09:50.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:50.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:50.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:50.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:49 smithi116 ceph-mon[89502]: pgmap v217: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s; 95 B/s, 5 objects/s recovering 2024-06-01T02:09:50.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:49 smithi116 ceph-mon[89502]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-01T02:09:50.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:49 smithi116 ceph-mon[89502]: Cluster is now healthy 2024-06-01T02:09:50.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:50.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:50.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:52.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:51 smithi161 ceph-mon[84406]: pgmap v218: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 41 B/s, 0 objects/s recovering 2024-06-01T02:09:52.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:51 smithi116 ceph-mon[89502]: pgmap v218: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 41 B/s, 0 objects/s recovering 2024-06-01T02:09:52.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:51 smithi116 ceph-mon[100606]: pgmap v218: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 41 B/s, 0 objects/s recovering 2024-06-01T02:09:54.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:53 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:09:54.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:53 smithi116 ceph-mon[100606]: pgmap v219: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 33 B/s, 0 objects/s recovering 2024-06-01T02:09:54.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:54.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:54.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:54.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:53 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:09:54.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:53 smithi116 ceph-mon[89502]: pgmap v219: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 33 B/s, 0 objects/s recovering 2024-06-01T02:09:54.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:54.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:54.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:54.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:53 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:09:54.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:53 smithi161 ceph-mon[84406]: pgmap v219: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 33 B/s, 0 objects/s recovering 2024-06-01T02:09:54.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:54.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:54.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:56.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:55 smithi161 ceph-mon[84406]: pgmap v220: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 940 B/s rd, 0 op/s 2024-06-01T02:09:56.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:55 smithi116 ceph-mon[89502]: pgmap v220: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 940 B/s rd, 0 op/s 2024-06-01T02:09:56.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:55 smithi116 ceph-mon[100606]: pgmap v220: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 940 B/s rd, 0 op/s 2024-06-01T02:09:57.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:57.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:56 smithi161 ceph-mon[84406]: pgmap v221: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:09:57.195 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:09:56 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:09:56] "GET /metrics HTTP/1.1" 200 38527 "" "Prometheus/2.33.4" 2024-06-01T02:09:57.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:57.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:56 smithi116 ceph-mon[89502]: pgmap v221: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:09:57.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:09:57.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:56 smithi116 ceph-mon[100606]: pgmap v221: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:09:59.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:09:59 smithi116 ceph-mon[89502]: pgmap v222: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:09:59.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:09:59 smithi116 ceph-mon[100606]: pgmap v222: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:09:59.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:09:59 smithi161 ceph-mon[84406]: pgmap v222: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:10:00.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:00 smithi116 ceph-mon[89502]: overall HEALTH_OK 2024-06-01T02:10:00.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:00 smithi116 ceph-mon[100606]: overall HEALTH_OK 2024-06-01T02:10:00.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:00 smithi161 ceph-mon[84406]: overall HEALTH_OK 2024-06-01T02:10:02.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:01 smithi161 ceph-mon[84406]: pgmap v223: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:10:02.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:01 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:02.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:01 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:10:02.212 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:01 smithi116 ceph-mon[89502]: pgmap v223: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:10:02.213 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:02.213 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:10:02.213 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:01 smithi116 ceph-mon[100606]: pgmap v223: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:10:02.213 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:01 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:02.213 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:01 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:10:03.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:02 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-01T02:10:03.290 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-01T02:10:03.290 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:02 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-01T02:10:04.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:03 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:10:04.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:03 smithi161 ceph-mon[84406]: pgmap v224: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:10:04.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:03 smithi161 ceph-mon[84406]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-01T02:10:04.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:03 smithi161 ceph-mon[84406]: Upgrade: osd.5 is safe to restart 2024-06-01T02:10:04.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:03 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:10:04.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:03 smithi116 ceph-mon[89502]: pgmap v224: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:10:04.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:03 smithi116 ceph-mon[89502]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-01T02:10:04.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:03 smithi116 ceph-mon[89502]: Upgrade: osd.5 is safe to restart 2024-06-01T02:10:04.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:03 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:10:04.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:03 smithi116 ceph-mon[100606]: pgmap v224: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:10:04.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:03 smithi116 ceph-mon[100606]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-01T02:10:04.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:03 smithi116 ceph-mon[100606]: Upgrade: osd.5 is safe to restart 2024-06-01T02:10:04.735 INFO:teuthology.orchestra.run.smithi116.stdout:true 2024-06-01T02:10:05.384 INFO:teuthology.orchestra.run.smithi116.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-01T02:10:05.385 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi116 smithi116 running 12s ago 18m - - 2024-06-01T02:10:05.385 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi161 smithi161 running 16s ago 18m - - 2024-06-01T02:10:05.385 INFO:teuthology.orchestra.run.smithi116.stdout:alertmanager.a smithi116 *:9093,9094 running (8m) 12s ago 13m 21.7M - 0.23.0 ba2b418f427c e0c297162ded 2024-06-01T02:10:05.385 INFO:teuthology.orchestra.run.smithi116.stdout:grafana.a smithi161 *:3000 running (7m) 16s ago 12m 51.7M - 8.3.5 dad864ee21e9 00b0a78ee88e 2024-06-01T02:10:05.385 INFO:teuthology.orchestra.run.smithi116.stdout:iscsi.foo.smithi116.gjuxzb smithi116 running (6m) 12s ago 11m 60.8M - 3.5 e1d6a67b021e 0e191fe3e3bb 2024-06-01T02:10:05.385 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.x smithi161 *:8443,9283,8765 running (7m) 16s ago 17m 528M - 19.0.0-2946-g3d89accf b045ac545f82 2d7fbb85e5f7 2024-06-01T02:10:05.385 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.y smithi116 *:8443,9283,8765 running (6m) 12s ago 19m 455M - 19.0.0-2946-g3d89accf b045ac545f82 bc9627c27a57 2024-06-01T02:10:05.385 INFO:teuthology.orchestra.run.smithi116.stdout:mon.a smithi116 running (5m) 12s ago 19m 68.9M 2048M 19.0.0-2946-g3d89accf b045ac545f82 b5e9b6f873ff 2024-06-01T02:10:05.385 INFO:teuthology.orchestra.run.smithi116.stdout:mon.b smithi161 running (4m) 16s ago 18m 43.2M 2048M 19.0.0-2946-g3d89accf b045ac545f82 aaa08c03cfc8 2024-06-01T02:10:05.385 INFO:teuthology.orchestra.run.smithi116.stdout:mon.c smithi116 running (4m) 12s ago 18m 50.4M 2048M 19.0.0-2946-g3d89accf b045ac545f82 bd0fb0e603fa 2024-06-01T02:10:05.385 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.a smithi116 *:9100 running (8m) 12s ago 13m 17.4M - 1.3.1 1dbe0e931976 e347748d17bb 2024-06-01T02:10:05.385 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.b smithi161 *:9100 running (7m) 16s ago 13m 19.8M - 1.3.1 1dbe0e931976 6e66182f5a1a 2024-06-01T02:10:05.385 INFO:teuthology.orchestra.run.smithi116.stdout:osd.0 smithi116 running (3m) 12s ago 17m 50.4M 4096M 19.0.0-2946-g3d89accf b045ac545f82 b4313d121b9d 2024-06-01T02:10:05.386 INFO:teuthology.orchestra.run.smithi116.stdout:osd.1 smithi116 running (2m) 12s ago 17m 52.7M 4096M 19.0.0-2946-g3d89accf b045ac545f82 dccdbeac5aae 2024-06-01T02:10:05.386 INFO:teuthology.orchestra.run.smithi116.stdout:osd.2 smithi116 running (113s) 12s ago 16m 46.5M 4096M 19.0.0-2946-g3d89accf b045ac545f82 1ad95efda404 2024-06-01T02:10:05.386 INFO:teuthology.orchestra.run.smithi116.stdout:osd.3 smithi116 running (82s) 12s ago 16m 50.4M 4096M 19.0.0-2946-g3d89accf b045ac545f82 bd00ad2c2556 2024-06-01T02:10:05.386 INFO:teuthology.orchestra.run.smithi116.stdout:osd.4 smithi161 running (32s) 16s ago 15m 49.1M 4096M 19.0.0-2946-g3d89accf b045ac545f82 0084b9edf7a0 2024-06-01T02:10:05.386 INFO:teuthology.orchestra.run.smithi116.stdout:osd.5 smithi161 running (15m) 16s ago 15m 61.2M 4096M 17.2.0 e1d6a67b021e 68358b78a25d 2024-06-01T02:10:05.386 INFO:teuthology.orchestra.run.smithi116.stdout:osd.6 smithi161 running (15m) 16s ago 15m 61.1M 4096M 17.2.0 e1d6a67b021e 50abeb0551e7 2024-06-01T02:10:05.386 INFO:teuthology.orchestra.run.smithi116.stdout:osd.7 smithi161 running (14m) 16s ago 14m 65.1M 4096M 17.2.0 e1d6a67b021e 6004e6983735 2024-06-01T02:10:05.386 INFO:teuthology.orchestra.run.smithi116.stdout:prometheus.a smithi161 *:9095 running (6m) 16s ago 14m 76.9M - 2.33.4 514e6a882f6e e977f6ca1d73 2024-06-01T02:10:05.387 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi116.vzjrxj smithi116 *:8000 running (12m) 12s ago 12m 77.4M - 17.2.0 e1d6a67b021e df4fe3c1dbc3 2024-06-01T02:10:05.387 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi161.ashqxz smithi161 *:8000 running (12m) 16s ago 12m 78.0M - 17.2.0 e1d6a67b021e 3eecc1f3e2d3 2024-06-01T02:10:05.387 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi116.hokbmf smithi116 *:80 running (11m) 12s ago 11m 77.7M - 17.2.0 e1d6a67b021e 10950ed75dbd 2024-06-01T02:10:05.387 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi161.axjlzw smithi161 *:80 running (11m) 16s ago 11m 77.0M - 17.2.0 e1d6a67b021e 0b5d5ce47772 2024-06-01T02:10:05.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:05 smithi161 ceph-mon[84406]: pgmap v225: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:10:05.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:05 smithi161 ceph-mon[84406]: Upgrade: Updating osd.5 2024-06-01T02:10:05.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:05.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-01T02:10:05.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:05.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:05 smithi161 ceph-mon[84406]: Deploying daemon osd.5 on smithi161 2024-06-01T02:10:05.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:05 smithi161 ceph-mon[84406]: from='client.44466 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:10:05.774 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:10:05.774 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": { 2024-06-01T02:10:05.774 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 3 2024-06-01T02:10:05.774 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:10:05.774 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": { 2024-06-01T02:10:05.774 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-01T02:10:05.774 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:10:05.774 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": { 2024-06-01T02:10:05.774 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 3, 2024-06-01T02:10:05.775 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 5 2024-06-01T02:10:05.775 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:10:05.775 INFO:teuthology.orchestra.run.smithi116.stdout: "rgw": { 2024-06-01T02:10:05.775 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-06-01T02:10:05.775 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:10:05.775 INFO:teuthology.orchestra.run.smithi116.stdout: "overall": { 2024-06-01T02:10:05.775 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 7, 2024-06-01T02:10:05.775 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 10 2024-06-01T02:10:05.775 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T02:10:05.775 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:10:05.775 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:05 smithi116 ceph-mon[100606]: pgmap v225: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:10:05.776 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:05 smithi116 ceph-mon[100606]: Upgrade: Updating osd.5 2024-06-01T02:10:05.776 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:05.776 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-01T02:10:05.776 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:05.776 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:05 smithi116 ceph-mon[100606]: Deploying daemon osd.5 on smithi161 2024-06-01T02:10:05.776 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:05 smithi116 ceph-mon[100606]: from='client.44466 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:10:05.776 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:05 smithi116 ceph-mon[89502]: pgmap v225: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:10:05.776 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:05 smithi116 ceph-mon[89502]: Upgrade: Updating osd.5 2024-06-01T02:10:05.776 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:05.776 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-01T02:10:05.777 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:05.777 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:05 smithi116 ceph-mon[89502]: Deploying daemon osd.5 on smithi161 2024-06-01T02:10:05.777 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:05 smithi116 ceph-mon[89502]: from='client.44466 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:10:06.108 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:10:06.108 INFO:teuthology.orchestra.run.smithi116.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-01T02:10:06.108 INFO:teuthology.orchestra.run.smithi116.stdout: "in_progress": true, 2024-06-01T02:10:06.108 INFO:teuthology.orchestra.run.smithi116.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-01T02:10:06.108 INFO:teuthology.orchestra.run.smithi116.stdout: "services_complete": [ 2024-06-01T02:10:06.109 INFO:teuthology.orchestra.run.smithi116.stdout: "mon", 2024-06-01T02:10:06.109 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr" 2024-06-01T02:10:06.109 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-06-01T02:10:06.109 INFO:teuthology.orchestra.run.smithi116.stdout: "progress": "10/23 daemons upgraded", 2024-06-01T02:10:06.109 INFO:teuthology.orchestra.run.smithi116.stdout: "message": "Currently upgrading osd daemons", 2024-06-01T02:10:06.109 INFO:teuthology.orchestra.run.smithi116.stdout: "is_paused": false 2024-06-01T02:10:06.109 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:10:06.536 INFO:teuthology.orchestra.run.smithi116.stdout:HEALTH_OK 2024-06-01T02:10:06.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:06 smithi161 ceph-mon[84406]: from='client.44472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:10:06.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:06 smithi161 ceph-mon[84406]: from='client.64444 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:10:06.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:06 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/1845880605' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:06.792 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:06 smithi116 ceph-mon[89502]: from='client.44472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:10:06.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:06 smithi116 ceph-mon[89502]: from='client.64444 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:10:06.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:06 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/1845880605' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:06.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:06 smithi116 ceph-mon[100606]: from='client.44472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:10:06.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:06 smithi116 ceph-mon[100606]: from='client.64444 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:10:06.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:06 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/1845880605' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:07.060 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:10:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:10:06] "GET /metrics HTTP/1.1" 200 38529 "" "Prometheus/2.33.4" 2024-06-01T02:10:07.581 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:07 smithi161 ceph-mon[84406]: from='client.64456 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:10:07.581 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:07 smithi161 ceph-mon[84406]: pgmap v226: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:10:07.581 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:07 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/1705121811' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:10:07.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:07 smithi116 ceph-mon[89502]: from='client.64456 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:10:07.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:07 smithi116 ceph-mon[89502]: pgmap v226: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:10:07.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:07 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/1705121811' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:10:07.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:07 smithi116 ceph-mon[100606]: from='client.64456 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:10:07.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:07 smithi116 ceph-mon[100606]: pgmap v226: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:10:07.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:07 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/1705121811' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:10:07.928 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:07 smithi161 systemd[1]: Stopping Ceph osd.5 for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:10:08.194 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[33652]: 2024-06-01T02:10:07.952+0000 7f14314fe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-01T02:10:08.194 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[33652]: 2024-06-01T02:10:07.952+0000 7f14314fe700 -1 osd.5 165 *** Got signal Terminated *** 2024-06-01T02:10:08.194 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[33652]: 2024-06-01T02:10:07.952+0000 7f14314fe700 -1 osd.5 165 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-01T02:10:08.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:08 smithi161 ceph-mon[84406]: osd.5 marked itself down and dead 2024-06-01T02:10:08.792 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:08 smithi116 ceph-mon[89502]: osd.5 marked itself down and dead 2024-06-01T02:10:08.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:08 smithi116 ceph-mon[100606]: osd.5 marked itself down and dead 2024-06-01T02:10:09.194 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:08 smithi161 podman[107510]: 2024-06-01 02:10:08.938969663 +0000 UTC m=+1.085635034 container died 68358b78a25de6a335028c249c53724b405f8e42f6218d379d06b19701b03df5 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5, architecture=x86_64, CEPH_POINT_RELEASE=-17.2.0, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, RELEASE=HEAD, io.openshift.expose-services=, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, GIT_CLEAN=True, version=8, maintainer=Guillaume Abrioux , com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, io.buildah.version=1.19.8, io.openshift.tags=base centos centos-stream, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, release=754, ceph=True, com.redhat.component=centos-stream-container, distribution-scope=public, vcs-type=git, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.k8s.display-name=CentOS Stream 8, build-date=2022-05-03T08:36:31.336870, name=centos-stream, vendor=Red Hat, Inc.) 2024-06-01T02:10:09.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:09 smithi161 ceph-mon[84406]: pgmap v227: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:10:09.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:09 smithi161 ceph-mon[84406]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:10:09.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:09 smithi161 ceph-mon[84406]: osdmap e166: 8 total, 7 up, 8 in 2024-06-01T02:10:09.695 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:09 smithi161 podman[107510]: 2024-06-01 02:10:09.277589738 +0000 UTC m=+1.424255156 container cleanup 68358b78a25de6a335028c249c53724b405f8e42f6218d379d06b19701b03df5 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5, build-date=2022-05-03T08:36:31.336870, io.openshift.tags=base centos centos-stream, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, release=754, GIT_CLEAN=True, architecture=x86_64, name=centos-stream, io.buildah.version=1.19.8, io.k8s.display-name=CentOS Stream 8, com.redhat.component=centos-stream-container, distribution-scope=public, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, GIT_BRANCH=HEAD, RELEASE=HEAD, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, maintainer=Guillaume Abrioux , vendor=Red Hat, Inc., GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, CEPH_POINT_RELEASE=-17.2.0, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, ceph=True, vcs-type=git, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., version=8, GIT_REPO=https://github.com/ceph/ceph-container.git, io.openshift.expose-services=) 2024-06-01T02:10:09.695 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:09 smithi161 bash[107510]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5 2024-06-01T02:10:09.695 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:09 smithi161 podman[107538]: 2024-06-01 02:10:09.453879476 +0000 UTC m=+0.514449681 container remove 68358b78a25de6a335028c249c53724b405f8e42f6218d379d06b19701b03df5 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, vcs-type=git, release=754, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, architecture=x86_64, maintainer=Guillaume Abrioux , summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, io.k8s.display-name=CentOS Stream 8, name=centos-stream, CEPH_POINT_RELEASE=-17.2.0, build-date=2022-05-03T08:36:31.336870, io.openshift.tags=base centos centos-stream, distribution-scope=public, vendor=Red Hat, Inc., io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., ceph=True, RELEASE=HEAD, GIT_CLEAN=True, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, io.openshift.expose-services=, GIT_REPO=https://github.com/ceph/ceph-container.git, com.redhat.component=centos-stream-container, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, version=8, io.buildah.version=1.19.8, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, GIT_BRANCH=HEAD, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb) 2024-06-01T02:10:09.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:09 smithi116 ceph-mon[89502]: pgmap v227: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:10:09.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:09 smithi116 ceph-mon[89502]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:10:09.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:09 smithi116 ceph-mon[89502]: osdmap e166: 8 total, 7 up, 8 in 2024-06-01T02:10:09.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:09 smithi116 ceph-mon[100606]: pgmap v227: 161 pgs: 161 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:10:09.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:09 smithi116 ceph-mon[100606]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:10:09.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:09 smithi116 ceph-mon[100606]: osdmap e166: 8 total, 7 up, 8 in 2024-06-01T02:10:10.194 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:09 smithi161 podman[107649]: 2024-06-01 02:10:09.782954541 +0000 UTC m=+0.017350263 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:10:10.194 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:09 smithi161 podman[107649]: 2024-06-01 02:10:09.904934156 +0000 UTC m=+0.139329870 container create bcc5feda5deb15390bb14f8065256eaee53841687b778ea1fb8a9530b30620c9 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=) 2024-06-01T02:10:10.194 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:10 smithi161 podman[107649]: 2024-06-01 02:10:10.116135036 +0000 UTC m=+0.350530758 container init bcc5feda5deb15390bb14f8065256eaee53841687b778ea1fb8a9530b30620c9 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2) 2024-06-01T02:10:10.194 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:10 smithi161 podman[107649]: 2024-06-01 02:10:10.119704924 +0000 UTC m=+0.354100655 container start bcc5feda5deb15390bb14f8065256eaee53841687b778ea1fb8a9530b30620c9 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-01T02:10:10.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:10 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:10.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:10 smithi161 ceph-mon[84406]: osdmap e167: 8 total, 7 up, 8 in 2024-06-01T02:10:10.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:10 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:10.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:10 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:10.695 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:10 smithi161 podman[107649]: 2024-06-01 02:10:10.207586673 +0000 UTC m=+0.441982409 container attach bcc5feda5deb15390bb14f8065256eaee53841687b778ea1fb8a9530b30620c9 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True) 2024-06-01T02:10:10.695 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:10 smithi161 podman[107649]: 2024-06-01 02:10:10.425365653 +0000 UTC m=+0.659761378 container died bcc5feda5deb15390bb14f8065256eaee53841687b778ea1fb8a9530b30620c9 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-01T02:10:10.792 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:10 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:10.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:10 smithi116 ceph-mon[89502]: osdmap e167: 8 total, 7 up, 8 in 2024-06-01T02:10:10.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:10 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:10.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:10 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:10.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:10 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:10.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:10 smithi116 ceph-mon[100606]: osdmap e167: 8 total, 7 up, 8 in 2024-06-01T02:10:10.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:10 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:10.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:10 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:11.436 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:11 smithi161 podman[107692]: 2024-06-01 02:10:11.021686795 +0000 UTC m=+0.590814679 container remove bcc5feda5deb15390bb14f8065256eaee53841687b778ea1fb8a9530b30620c9 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_CLEAN=True) 2024-06-01T02:10:11.436 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:11 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.5.service: Deactivated successfully. 2024-06-01T02:10:11.436 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:11 smithi161 systemd[1]: Stopped Ceph osd.5 for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:10:11.436 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:11 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.5.service: Consumed 10.598s CPU time. 2024-06-01T02:10:11.437 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:11 smithi161 systemd[1]: Starting Ceph osd.5 for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:10:11.693 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:11 smithi161 ceph-mon[84406]: pgmap v230: 161 pgs: 22 stale+active+clean, 139 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:10:11.792 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:11 smithi116 ceph-mon[89502]: pgmap v230: 161 pgs: 22 stale+active+clean, 139 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:10:11.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:11 smithi116 ceph-mon[100606]: pgmap v230: 161 pgs: 22 stale+active+clean, 139 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:10:12.191 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:12 smithi161 podman[107807]: 2024-06-01 02:10:11.997352865 +0000 UTC m=+0.017572698 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:10:12.191 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:12 smithi161 podman[107807]: 2024-06-01 02:10:12.103009562 +0000 UTC m=+0.123229433 container create e4076079f9798934b66a81c785e3ed5ca511648a6581e24f1918e0f88150c295 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-activate, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528) 2024-06-01T02:10:12.444 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:12 smithi161 podman[107807]: 2024-06-01 02:10:12.305446525 +0000 UTC m=+0.325666362 container init e4076079f9798934b66a81c785e3ed5ca511648a6581e24f1918e0f88150c295 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-activate, ceph=True, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD) 2024-06-01T02:10:12.444 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:12 smithi161 podman[107807]: 2024-06-01 02:10:12.308971597 +0000 UTC m=+0.329191426 container start e4076079f9798934b66a81c785e3ed5ca511648a6581e24f1918e0f88150c295 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2) 2024-06-01T02:10:12.444 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:12 smithi161 podman[107807]: 2024-06-01 02:10:12.388726887 +0000 UTC m=+0.408946746 container attach e4076079f9798934b66a81c785e3ed5ca511648a6581e24f1918e0f88150c295 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux ) 2024-06-01T02:10:12.758 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:12 smithi161 ceph-mon[84406]: Health check failed: Degraded data redundancy: 16/726 objects degraded (2.204%), 8 pgs degraded (PG_DEGRADED) 2024-06-01T02:10:12.792 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:12 smithi116 ceph-mon[100606]: Health check failed: Degraded data redundancy: 16/726 objects degraded (2.204%), 8 pgs degraded (PG_DEGRADED) 2024-06-01T02:10:12.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:12 smithi116 ceph-mon[89502]: Health check failed: Degraded data redundancy: 16/726 objects degraded (2.204%), 8 pgs degraded (PG_DEGRADED) 2024-06-01T02:10:13.079 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:12 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-activate[107822]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-06-01T02:10:13.079 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:12 smithi161 bash[107807]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-06-01T02:10:13.079 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:12 smithi161 bash[107807]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-06-01T02:10:13.079 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:12 smithi161 bash[107807]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-06-01T02:10:13.079 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:12 smithi161 bash[107807]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-06-01T02:10:13.079 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:12 smithi161 bash[107807]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-5/block 2024-06-01T02:10:13.079 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:12 smithi161 bash[107807]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-06-01T02:10:13.079 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:12 smithi161 bash[107807]: --> ceph-volume raw activate successful for osd ID: 5 2024-06-01T02:10:13.079 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:12 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-activate[107822]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-06-01T02:10:13.080 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:12 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-activate[107822]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-06-01T02:10:13.080 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:12 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-activate[107822]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-06-01T02:10:13.080 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:12 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-activate[107822]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-5/block 2024-06-01T02:10:13.080 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:12 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-activate[107822]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-06-01T02:10:13.080 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:12 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-activate[107822]: --> ceph-volume raw activate successful for osd ID: 5 2024-06-01T02:10:13.080 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:12 smithi161 podman[107807]: 2024-06-01 02:10:12.78810267 +0000 UTC m=+0.808322499 container died e4076079f9798934b66a81c785e3ed5ca511648a6581e24f1918e0f88150c295 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-activate, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-06-01T02:10:13.694 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:13 smithi161 podman[107954]: 2024-06-01 02:10:13.384512396 +0000 UTC m=+0.590800576 container remove e4076079f9798934b66a81c785e3ed5ca511648a6581e24f1918e0f88150c295 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-activate, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-06-01T02:10:13.694 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:13 smithi161 podman[108013]: 2024-06-01 02:10:13.498735738 +0000 UTC m=+0.016982280 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:10:13.694 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:13 smithi161 podman[108013]: 2024-06-01 02:10:13.610490173 +0000 UTC m=+0.128736766 container create 596f1b7846a2f9d7bcc47103ca40d1ec3f32b75da3fc0c254df90d2ea5635179 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-06-01T02:10:14.120 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:13 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:10:14.120 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:13 smithi161 ceph-mon[84406]: pgmap v231: 161 pgs: 8 active+undersized+degraded, 15 stale+active+clean, 11 active+undersized, 127 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 16/726 objects degraded (2.204%) 2024-06-01T02:10:14.120 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:14.121 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:14.121 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:14.121 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:13 smithi161 podman[108013]: 2024-06-01 02:10:13.829419555 +0000 UTC m=+0.347666083 container init 596f1b7846a2f9d7bcc47103ca40d1ec3f32b75da3fc0c254df90d2ea5635179 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-06-01T02:10:14.121 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:13 smithi161 podman[108013]: 2024-06-01 02:10:13.832589234 +0000 UTC m=+0.350835771 container start 596f1b7846a2f9d7bcc47103ca40d1ec3f32b75da3fc0c254df90d2ea5635179 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-06-01T02:10:14.121 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:13 smithi161 ceph-osd[108031]: -- 172.21.15.161:0/3869270840 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555d32629680 con 0x555d325f2c00 2024-06-01T02:10:14.121 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:13 smithi161 bash[108013]: 596f1b7846a2f9d7bcc47103ca40d1ec3f32b75da3fc0c254df90d2ea5635179 2024-06-01T02:10:14.121 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:13 smithi161 systemd[1]: Started Ceph osd.5 for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:10:14.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:13 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:10:14.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:13 smithi116 ceph-mon[89502]: pgmap v231: 161 pgs: 8 active+undersized+degraded, 15 stale+active+clean, 11 active+undersized, 127 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 16/726 objects degraded (2.204%) 2024-06-01T02:10:14.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:14.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:14.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:14.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:13 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:10:14.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:13 smithi116 ceph-mon[100606]: pgmap v231: 161 pgs: 8 active+undersized+degraded, 15 stale+active+clean, 11 active+undersized, 127 active+clean; 585 KiB data, 209 MiB used, 715 GiB / 715 GiB avail; 16/726 objects degraded (2.204%) 2024-06-01T02:10:14.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:14.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:14.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:14.372 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[108027]: 2024-06-01T02:10:14.373+0000 7fbb206447c0 -1 Falling back to public interface 2024-06-01T02:10:15.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:15.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:15.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:15.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:15.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:15.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:15.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:15.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:15.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:15.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:15.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:15.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:15.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: pgmap v232: 161 pgs: 23 active+undersized+degraded, 34 active+undersized, 104 active+clean; 585 KiB data, 210 MiB used, 715 GiB / 715 GiB avail; 81/726 objects degraded (11.157%) 2024-06-01T02:10:15.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:15.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:15.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:15.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:15.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:15.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:15.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:15.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:15.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:15.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:15.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:15.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:15.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:15.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:15.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:15.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:15.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:15.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:15.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:15.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:15.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:15.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:15.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:15.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:15.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:15.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:15.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:15.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:15.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:15.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:14 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:15.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:15.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:15.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:15.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:15.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:15.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:15.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:15.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:15.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:15.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:15.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:15.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:15.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: pgmap v232: 161 pgs: 23 active+undersized+degraded, 34 active+undersized, 104 active+clean; 585 KiB data, 210 MiB used, 715 GiB / 715 GiB avail; 81/726 objects degraded (11.157%) 2024-06-01T02:10:15.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:15.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:15.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:15.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:15.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:15.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:15.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:15.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:15.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:15.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:15.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:15.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:15.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:15.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:15.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:15.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:15.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:15.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:15.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:15.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:15.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:15.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:15.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:15.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:15.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:15.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:15.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:15.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: pgmap v232: 161 pgs: 23 active+undersized+degraded, 34 active+undersized, 104 active+clean; 585 KiB data, 210 MiB used, 715 GiB / 715 GiB avail; 81/726 objects degraded (11.157%) 2024-06-01T02:10:15.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:15.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:15.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:15.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:15.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:15.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:15.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:15.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:15.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:15.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:15.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:15.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:15.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:15.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:15.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:15.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:15.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:15.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:15.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:15.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:15.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:15.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:15.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:15.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:15.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:15.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:15.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:15.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:15.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:15.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:15.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:15.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:15.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:15.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:15.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:15.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:15.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:15.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:15.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:15.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:15.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:15.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:15.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:15.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:15.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:14 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:16.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:16.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:16.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:16.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:16.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:16.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:16.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:16.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:16.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:16.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:16.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:16.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:16.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:16.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:16.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:16.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:16.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:16.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:16.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:16.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:16.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:16.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:16.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:16.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:16.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:16.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:16.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:16.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:16.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:16.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:16.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:16.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:16.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:16.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:16.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:16.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:16.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:16.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:16.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:16.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:16.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:16.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:16.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:16.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:16.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:16.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:16.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:16.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:16.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:16.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:10:16.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:16.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:16.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:16.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:16.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:16.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:15 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:16.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:16.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:16.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:16.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:16.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:16.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:16.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:16.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:16.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:16.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:16.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:16.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:16.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:16.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:16.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:16.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:16.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:16.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:16.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:16.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:16.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:16.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:16.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:16.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:16.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:16.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:16.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:16.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:16.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:16.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:16.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:16.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:16.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:16.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:16.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:16.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:16.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:16.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:16.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:16.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:16.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:16.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:16.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:16.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:16.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:16.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:16.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:16.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:16.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:16.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:16.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:16.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:16.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:16.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:16.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:16.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:16.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:16.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:16.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:16.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:16.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:16.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:16.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:16.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:16.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:16.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:16.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:16.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:16.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:16.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:16.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:16.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:16.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:16.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:16.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:16.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:16.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:16.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:16.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:10:16.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:16.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:16.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:16.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:16.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:16.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:16.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:16.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:16.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:16.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:16.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:16.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:16.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:16.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:16.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:16.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:16.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:16.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:16.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:16.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:16.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:16.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:16.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:16.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:16.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:16.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:16.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:10:16.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:16.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:16.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:16.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:16.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:16.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:15 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:17.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:17.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:17.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:17.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:17.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:17.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:17.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:17.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:17.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:17.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:17.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:17.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:17.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: pgmap v233: 161 pgs: 23 active+undersized+degraded, 34 active+undersized, 104 active+clean; 585 KiB data, 210 MiB used, 715 GiB / 715 GiB avail; 81/726 objects degraded (11.157%) 2024-06-01T02:10:17.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:17.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:17.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:17.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:17.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:17.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:17.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:17.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:17.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:17.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:17.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:17.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:17.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:17.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:17.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:17.087 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:10:16] "GET /metrics HTTP/1.1" 200 38531 "" "Prometheus/2.33.4" 2024-06-01T02:10:17.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:17.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:17.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:17.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:17.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:17.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:17.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:17.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:17.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:17.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:17.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:17.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:17.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:17.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:17.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:17.089 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:17.089 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:17.089 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:17.089 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:17.089 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:17.089 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:16 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:17.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:17.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:17.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:17.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:17.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:17.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:17.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:17.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:17.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:17.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:17.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:17.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:17.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: pgmap v233: 161 pgs: 23 active+undersized+degraded, 34 active+undersized, 104 active+clean; 585 KiB data, 210 MiB used, 715 GiB / 715 GiB avail; 81/726 objects degraded (11.157%) 2024-06-01T02:10:17.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:17.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:17.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:17.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:17.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:17.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:17.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:17.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:17.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:17.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:17.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:17.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:17.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:17.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:17.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:17.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:17.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:17.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:17.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:17.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:17.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:17.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:17.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:17.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:17.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:17.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:17.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:17.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: pgmap v233: 161 pgs: 23 active+undersized+degraded, 34 active+undersized, 104 active+clean; 585 KiB data, 210 MiB used, 715 GiB / 715 GiB avail; 81/726 objects degraded (11.157%) 2024-06-01T02:10:17.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:17.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:17.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:17.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:17.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:17.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:17.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:17.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:17.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:17.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:17.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:17.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:17.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:17.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:17.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:17.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:17.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:17.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:17.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:17.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:17.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:17.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:17.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:17.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:17.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:17.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:17.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:17.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:17.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:17.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:17.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:17.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:17.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:17.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:17.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:17.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:17.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:17.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:17.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:17.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:17.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:17.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:17.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:17.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:17.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:17.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:17.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:17.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:17.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:17.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:17.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:17.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:17.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:17.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:17.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:17.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:17.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:16 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:17.444 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:17 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[108027]: 2024-06-01T02:10:17.095+0000 7fbb206447c0 -1 osd.5 0 read_superblock omap replica is missing. 2024-06-01T02:10:17.444 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:17 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[108027]: 2024-06-01T02:10:17.135+0000 7fbb206447c0 -1 osd.5 165 log_to_monitors true 2024-06-01T02:10:18.277 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[108027]: 2024-06-01T02:10:18.106+0000 7fbb11a9b700 -1 osd.5 165 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-01T02:10:18.277 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:18.277 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:18.277 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:18.277 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:18.277 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:18.278 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:17 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:18.278 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:17 smithi161 ceph-mon[84406]: from='osd.5 [v2:172.21.15.161:6808/3116494623,v1:172.21.15.161:6809/3116494623]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-01T02:10:18.278 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:17 smithi161 ceph-mon[84406]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-01T02:10:18.278 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:18.278 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:18.278 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:18.278 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:18.278 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:18.278 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:17 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:18.278 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:18.278 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:18.278 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:18.279 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:18.279 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:18.279 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:18.279 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:18.279 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:18.279 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:18.279 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:18.279 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:18.279 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:18.279 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:18.279 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:18.279 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:18.279 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:18.279 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:18.280 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:18.280 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:18.280 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:18.280 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:18.280 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:18.280 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:18.280 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:18.280 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:18.280 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:18.280 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:18.280 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:18.280 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:18.280 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:18.280 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:18.281 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:18.281 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:18.281 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:18.281 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:18.281 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:18 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:18.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:18.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:18.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:18.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:18.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:18.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:17 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:18.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:17 smithi116 ceph-mon[89502]: from='osd.5 [v2:172.21.15.161:6808/3116494623,v1:172.21.15.161:6809/3116494623]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-01T02:10:18.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:17 smithi116 ceph-mon[89502]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-01T02:10:18.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:18.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:18.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:18.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:18.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:18.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:17 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:18.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:18.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:18.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:18.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:18.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:18.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:18.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:18.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:18.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:18.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:18.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:18.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:18.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:18.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:18.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:18.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:18.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:18.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:18.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:18.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:18.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:18.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:18.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:18.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:18.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:18.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:18.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:18.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:18.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:18.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:18.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:18.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:18.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:18.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:18.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:18.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:18.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:18.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:18.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:18.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:18.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:18.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:17 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:18.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:17 smithi116 ceph-mon[100606]: from='osd.5 [v2:172.21.15.161:6808/3116494623,v1:172.21.15.161:6809/3116494623]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-01T02:10:18.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:17 smithi116 ceph-mon[100606]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-01T02:10:18.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:18.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:18.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:18.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:18.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:18.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:18.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:18.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:18.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:18.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:18.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:18.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:18.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:18.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:18.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:18.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:18.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:18.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:18.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:18.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:18.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:18.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:18.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:18.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:18.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:18.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:18.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:18.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:18.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:18.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:18.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:18.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:18.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:18.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:18.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:18.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:18.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:18.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:18.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:18.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:18.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:18.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:18 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:19.326 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-01T02:10:19.326 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: osdmap e168: 8 total, 7 up, 8 in 2024-06-01T02:10:19.326 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='osd.5 [v2:172.21.15.161:6808/3116494623,v1:172.21.15.161:6809/3116494623]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:10:19.326 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:10:19.326 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:19.327 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:19.327 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:19.327 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:19.327 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:19.327 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:19.327 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: pgmap v235: 161 pgs: 23 active+undersized+degraded, 34 active+undersized, 104 active+clean; 585 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 81/726 objects degraded (11.157%) 2024-06-01T02:10:19.327 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:19.327 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:19.327 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:19.327 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:19.327 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:19.327 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:19.327 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:19.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:19.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:19.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:19.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:19.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:19.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:19.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:19.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:19.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:19.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:19.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:19.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:19.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:19.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:19.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:19.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:19.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:19.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:19.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:19.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:19.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:19.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:19.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:19.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:19.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:19.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:19.330 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:19.330 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:19.330 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:19.330 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:19.330 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:19.330 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:19.330 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:19.330 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:19.330 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:19.330 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:19.330 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:19.330 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:19.331 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:19.331 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:19 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:19.394 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-01T02:10:19.394 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: osdmap e168: 8 total, 7 up, 8 in 2024-06-01T02:10:19.394 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='osd.5 [v2:172.21.15.161:6808/3116494623,v1:172.21.15.161:6809/3116494623]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:10:19.394 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:10:19.394 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:19.394 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:19.394 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:19.394 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:19.395 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:19.395 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:19.395 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: pgmap v235: 161 pgs: 23 active+undersized+degraded, 34 active+undersized, 104 active+clean; 585 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 81/726 objects degraded (11.157%) 2024-06-01T02:10:19.395 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:19.395 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:19.395 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:19.395 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:19.395 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:19.395 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:19.395 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:19.395 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:19.395 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:19.395 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:19.396 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:19.396 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:19.396 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:19.396 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:19.396 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:19.396 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-01T02:10:19.396 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: osdmap e168: 8 total, 7 up, 8 in 2024-06-01T02:10:19.396 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='osd.5 [v2:172.21.15.161:6808/3116494623,v1:172.21.15.161:6809/3116494623]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:10:19.396 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:10:19.396 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:19.396 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:19.396 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:19.397 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:19.397 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:19.397 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:19.397 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: pgmap v235: 161 pgs: 23 active+undersized+degraded, 34 active+undersized, 104 active+clean; 585 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 81/726 objects degraded (11.157%) 2024-06-01T02:10:19.397 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:19.397 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:19.397 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:19.397 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:19.397 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:19.397 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:19.397 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:19.397 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:19.397 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:19.397 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:19.397 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:19.398 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:19.398 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:19.398 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:19.398 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:19.398 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:19.398 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:19.398 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:19.398 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:19.399 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:19.399 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:19.399 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:19.399 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:19.399 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:19.399 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:19.399 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:19.399 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:19.399 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:19.399 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:19.399 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:19.399 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:19.399 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:19.399 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:19.399 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:19.400 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:19.400 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:19.400 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:19.400 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:19.400 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:19.400 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:19.400 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:19.400 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:19.400 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:19.400 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:19.400 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:19.400 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:19.400 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:19.400 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:19.401 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:19.401 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:19.401 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:19.401 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:19.401 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:19.401 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:19.401 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:19.401 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:19.401 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:19.401 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:19.402 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:19.402 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:19.402 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:19.402 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:19.402 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:19.402 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:19.402 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:19.402 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:19.402 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:19.402 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:19.402 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:19.402 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:19.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:19.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:19.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:19.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:19.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:19.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:19.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:19.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:19.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:19 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:20.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:20.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:10:20.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: osd.5 [v2:172.21.15.161:6808/3116494623,v1:172.21.15.161:6809/3116494623] boot 2024-06-01T02:10:20.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: osdmap e169: 8 total, 8 up, 8 in 2024-06-01T02:10:20.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T02:10:20.337 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:20.337 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:20.337 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:20.337 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:20.337 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:20.337 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:20.337 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:20.337 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:20.337 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:20.337 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:20.337 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:20.337 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:20.337 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:20.337 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:20.338 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:20.338 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:20.338 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:20.338 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:20.338 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:20.338 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:20.338 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:20.338 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:20.338 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:20.338 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:20.338 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:20.338 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:20.338 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:20.339 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:20.339 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:20.339 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:20.339 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:20.339 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:20.339 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:20.339 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:20.339 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:20.339 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:20.339 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:20.339 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:20.339 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:20.339 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:20.340 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:20.340 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:20.340 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:20.340 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:20.340 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:20.340 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:20.340 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:20.340 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:20.340 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:20.340 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:20.340 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:20.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:20.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:10:20.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: osd.5 [v2:172.21.15.161:6808/3116494623,v1:172.21.15.161:6809/3116494623] boot 2024-06-01T02:10:20.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: osdmap e169: 8 total, 8 up, 8 in 2024-06-01T02:10:20.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T02:10:20.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:20.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:20.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:20.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:20.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:20.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:20.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:20.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:20.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:20.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:20.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:20.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:20.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:20.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:20.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:20.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:20.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:20.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:20.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:20.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:20.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:20.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:20.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:20.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:20.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:20.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:20.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:10:20.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: osd.5 [v2:172.21.15.161:6808/3116494623,v1:172.21.15.161:6809/3116494623] boot 2024-06-01T02:10:20.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: osdmap e169: 8 total, 8 up, 8 in 2024-06-01T02:10:20.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T02:10:20.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:20.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:20.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:20.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:20.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:20.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:20.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:20.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:20.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:20.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:20.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:20.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:20.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:20.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:20.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:20.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:20.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:20.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:20.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:20.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:20.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:20.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:20.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:20.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:20.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:20.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:20.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:20.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:20.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:20.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:20.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:20.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:20.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:20.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:20.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:20.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:20.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:20.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:20.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:20.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:20.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:20.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:20.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:20.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:20.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:20.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:20.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:20.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:20.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:20.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:20.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:20.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:20.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:20.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:20.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:20.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:20.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:20.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:20.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:20.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:20.550 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:20.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:20.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:20.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:20.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:20.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:20.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:20.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:20.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:20.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:20.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:20.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:20.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:20.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:20.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:20.551 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:20.552 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:21.350 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:21.350 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:21.350 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:21.350 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: osdmap e170: 8 total, 8 up, 8 in 2024-06-01T02:10:21.350 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:21.350 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:21.351 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:21.351 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:21.351 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:21.351 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:21.351 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: pgmap v238: 161 pgs: 23 active+undersized+degraded, 34 active+undersized, 104 active+clean; 585 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 81/726 objects degraded (11.157%) 2024-06-01T02:10:21.351 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:21.351 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:21.351 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:21.351 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:21.351 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:21.351 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:21.351 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:21.352 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:21.352 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:21.352 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:21.352 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:21.352 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:21.352 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:21.352 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:21.352 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:21.352 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:21.352 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:21.352 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:21.353 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:21.353 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:21.353 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:21.353 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: osdmap e170: 8 total, 8 up, 8 in 2024-06-01T02:10:21.353 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:21.353 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:21.353 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:21.353 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:21.353 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:21.353 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:21.353 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: pgmap v238: 161 pgs: 23 active+undersized+degraded, 34 active+undersized, 104 active+clean; 585 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 81/726 objects degraded (11.157%) 2024-06-01T02:10:21.353 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:21.353 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:21.354 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:21.354 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:21.354 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:21.354 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:21.354 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:21.354 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:21.354 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:21.354 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:21.354 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:21.354 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:21.354 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:21.354 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:21.354 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:21.355 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:21.355 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:21.355 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:21.355 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:21.355 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:21.355 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:21.355 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:21.355 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:21.355 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:21.355 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:21.356 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:21.356 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:21.356 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:21.356 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:21.356 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:21.356 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:21.356 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:21.356 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:21.356 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:21.356 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:21.356 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:21.356 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:21.356 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:21.356 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:21.357 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:21.357 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:21.357 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:21.357 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:21.357 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:21.357 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:21.357 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:21.357 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:21.357 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:21.357 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:21.357 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:21.358 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:21.358 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:21.358 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:21.358 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:21.358 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:21.358 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:21.358 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:21.358 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:21.358 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:21.358 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:21.358 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:21.358 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:21.358 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:21.359 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:21.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:21.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:21.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:21.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: osdmap e170: 8 total, 8 up, 8 in 2024-06-01T02:10:21.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:21.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:21.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:21.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:21.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:21.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:21.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: pgmap v238: 161 pgs: 23 active+undersized+degraded, 34 active+undersized, 104 active+clean; 585 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 81/726 objects degraded (11.157%) 2024-06-01T02:10:21.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:21.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:21.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:21.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:21.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:21.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:21.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:21.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:21.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:21.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:21.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:21.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:21.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:21.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:21.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:21.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:21.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:21.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:21.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:21.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:21.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:21.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:21.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:21.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:21.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:21.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:21.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:21.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:21.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:21.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:21.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:21.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:21.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:21.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:21.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:21.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:21.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:21.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:21.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:21.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:21.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:22.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:22.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:22.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:22.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:22.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:22.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:22.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:22.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:22.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:22.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:22.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:22.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:22.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:22.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: Health check update: Degraded data redundancy: 81/726 objects degraded (11.157%), 23 pgs degraded (PG_DEGRADED) 2024-06-01T02:10:22.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:22.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:22.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:22.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:22.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:22.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:22.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:22.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:22.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:22.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:22.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:22.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:22.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:22.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:22.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:22.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:22.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:22.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:22.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:22.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:22.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:22.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:22.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:22.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:22.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:22.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:22.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:22.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:22.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:22.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:22.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:22.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:22.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:22.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:22.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:22.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:22.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:22.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:22.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:22.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:22.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:22.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:22.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: Health check update: Degraded data redundancy: 81/726 objects degraded (11.157%), 23 pgs degraded (PG_DEGRADED) 2024-06-01T02:10:22.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:22.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:22.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:22.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:22.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:22.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:22.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:22.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:22.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:22.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:22.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:22.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:22.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:22.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:22.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:22.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:22.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:22.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:22.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:22.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:22.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:22.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:22.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:22.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:22.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:22.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:22.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:22.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:22.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:22.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:22.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:22.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:22.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:22.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:22.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:22.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:22.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:22.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:22.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:22.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:22.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:22.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:22.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: Health check update: Degraded data redundancy: 81/726 objects degraded (11.157%), 23 pgs degraded (PG_DEGRADED) 2024-06-01T02:10:22.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:22.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:22.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:22.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:22.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:22.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:22.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:22.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:22.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:22.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:22.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:22.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:22.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:22.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:22.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:22.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:22.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:22.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:23.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:23.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:23.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:10:23.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:23.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:23.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:23.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:23.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:23.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:23.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:23.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:23.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:23.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:23.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:23.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: pgmap v239: 161 pgs: 18 active+undersized+degraded, 27 active+undersized, 116 active+clean; 585 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 68/726 objects degraded (9.366%) 2024-06-01T02:10:23.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:23.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:23.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:23.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:23.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:23.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:23.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:23.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:23.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:23.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:23.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:23.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:23.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:23.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:23.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:23.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:23.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:23.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:23.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:23.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:23.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:23.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:23.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:23.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:23.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:23.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:23.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:23.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:23.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:23.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:23.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:23.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:23.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:23.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:23.448 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:23.516 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:23.516 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:23.516 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:10:23.516 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:23.516 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:23.516 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:23.516 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:23.516 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:23.516 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:23.516 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:23.516 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:23.516 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:23.517 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:23.517 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:23.517 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: pgmap v239: 161 pgs: 18 active+undersized+degraded, 27 active+undersized, 116 active+clean; 585 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 68/726 objects degraded (9.366%) 2024-06-01T02:10:23.517 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:23.517 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:23.517 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:23.517 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:23.517 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:23.517 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:23.517 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:23.517 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:23.517 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:23.517 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:23.517 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:23.517 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:23.517 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:23.518 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:23.518 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:23.518 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:10:23.518 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:23.518 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:23.518 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:23.518 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:23.518 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:23.518 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:23.518 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:23.519 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:23.519 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:23.519 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:23.519 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:23.519 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: pgmap v239: 161 pgs: 18 active+undersized+degraded, 27 active+undersized, 116 active+clean; 585 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 68/726 objects degraded (9.366%) 2024-06-01T02:10:23.519 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:23.519 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:23.519 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:23.519 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:23.519 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:23.519 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:23.519 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:23.519 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:23.519 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:23.520 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:23.520 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:23.520 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:23.520 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:23.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:23.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:23.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:23.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:23.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:23.520 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:23.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:23.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:23.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:23.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:23.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:23.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:23.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:23.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:23.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:23.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:23.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:23.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:23.521 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:23.522 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:23.522 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:23.522 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:23.522 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:23.522 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:23.522 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:23.522 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:23.522 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:23.522 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:23.523 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:23.523 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:23.523 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:23.523 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:23.523 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:23.523 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:23.523 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:23.523 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:23.523 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:23.523 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:23.523 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:23.523 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:23.523 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:23.523 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:23.524 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:23.524 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:24.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:24.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:24.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:24.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:24.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:24.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:24.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:24.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:24.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:24.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:24.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:24.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:24.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:24.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:24.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:24.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:24.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:24.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:24.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:24.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:24.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:24.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:24.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:24.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:24.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:24.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:24.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:24.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:24.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:24.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:24.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:24.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:24.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:24.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:24.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:24.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:24.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:24.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:24.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:24.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:24.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:24.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:24.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:24.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:24.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:24.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:24.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:24.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:24.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:24.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:24.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:24.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:24.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:24.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:24.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:24.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:24.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:24.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:24.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:24.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:24.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:24.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:24.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:24.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:24.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:24.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:24.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:24.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:24.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:24.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:24.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:24.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:24.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:24.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:24.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:24.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:24.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:24.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:24.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:24.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:24.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:24.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:24.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:24.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:24.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:24.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:24.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:24.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:24.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:24.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:24.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:24.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:24.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:24.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:24.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:24.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:24.364 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:24.364 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:24.364 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:24.364 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:24.364 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:24.364 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:24.364 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:24.364 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:24.364 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:24.364 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:24.365 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:24.365 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:24.365 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:24.365 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:24.365 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:24.365 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:24.365 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:24.365 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:24.365 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:24.365 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:24.365 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:24.365 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:24.365 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:24.365 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:24.366 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:24.366 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:24.366 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:24.366 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:24.366 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:24.366 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:24.366 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:24.366 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:24.366 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:24.366 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:24.366 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:24.366 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:24.366 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:24.367 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:24.367 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:24.367 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:24.367 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:24.367 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:24.367 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:24.367 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:24.367 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:24.367 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:24.367 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:24.367 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:25.400 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:25.401 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:25.401 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:25.401 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:25.401 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:25.401 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:25.401 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:25.401 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: pgmap v240: 161 pgs: 161 active+clean; 585 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 332 B/s rd, 0 op/s 2024-06-01T02:10:25.401 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:25.401 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:25.401 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:25.401 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:25.401 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:25.401 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:25.401 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:25.402 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:25.402 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:25.402 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:25.402 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:25.402 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:25.402 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:25.402 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:25.402 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:25.402 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:25.402 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:25.402 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:25.402 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:25.402 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:25.403 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:25.403 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:25.403 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:25.403 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:25.403 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:25.403 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:25.403 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:25.403 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:25.403 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:25.403 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:25.403 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:25.403 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:25.403 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:25.403 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:25.404 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:25.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:25.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:25.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:25.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:25.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:25.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:25.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:25.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: pgmap v240: 161 pgs: 161 active+clean; 585 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 332 B/s rd, 0 op/s 2024-06-01T02:10:25.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:25.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:25.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:25.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:25.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:25.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:25.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:25.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:25.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:25.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:25.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:25.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:25.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:25.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:25.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:25.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:25.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:25.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:25.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:25.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:25.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:25.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:25.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:25.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:25.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:25.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:25.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:25.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:25.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: pgmap v240: 161 pgs: 161 active+clean; 585 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 332 B/s rd, 0 op/s 2024-06-01T02:10:25.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:25.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:25.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:25.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:25.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:25.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:25.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:25.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:25.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:25.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:25.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:25.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:25.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:25.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:25.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:25.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:25.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:25.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:25.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:25.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:25.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:25.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:25.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:25.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:25.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:25.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:25.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:25.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:25.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:25.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:25.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:25.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:25.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:25.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:25.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:25.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:25.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:25.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:25.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:25.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:25.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:25.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:25.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:25.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:25.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:25.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:25.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:25.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:25.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:26.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:26.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:26.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:26.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:26.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:26.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:26.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:26.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 68/726 objects degraded (9.366%), 18 pgs degraded) 2024-06-01T02:10:26.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: Cluster is now healthy 2024-06-01T02:10:26.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:26.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:26.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:26.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:26.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:26.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:26.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:26.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:26.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:26.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:26.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:26.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:26.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:26.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:26.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:26.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:26.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:26.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:26.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:26.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:26.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:26.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:26.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:26.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:26.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:26.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:26.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:26.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:26.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:26.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:26.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:26.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:26.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:26.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:26.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:26.448 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:26.448 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:26.448 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:26.448 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:26.448 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:26.448 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:26.448 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:26.448 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 68/726 objects degraded (9.366%), 18 pgs degraded) 2024-06-01T02:10:26.448 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: Cluster is now healthy 2024-06-01T02:10:26.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:26.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:26.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:26.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:26.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:26.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:26.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:26.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:26.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:26.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:26.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:26.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:26.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:26.450 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:26.450 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:26.450 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:26.450 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:26.450 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:26.450 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:26.450 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:26.450 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:26.450 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:26.450 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:26.451 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:26.451 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:26.451 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:26.451 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:26.451 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 68/726 objects degraded (9.366%), 18 pgs degraded) 2024-06-01T02:10:26.451 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: Cluster is now healthy 2024-06-01T02:10:26.451 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:26.451 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:26.451 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:26.451 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:26.451 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:26.451 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:26.451 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:26.451 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:26.451 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:26.452 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:26.453 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:26.453 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:26.453 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:26.454 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:26.454 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:26.454 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:26.454 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:26.454 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:26.454 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:26.454 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:26.454 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:26.454 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:26.454 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:26.455 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:26.455 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:26.455 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:26.455 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:26.455 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:26.455 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:26.455 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:26.455 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:26.455 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:26.455 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:26.455 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:26.455 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:26.455 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:26.456 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:26.456 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:26.456 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:26.456 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:26.456 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:26.456 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:26.456 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:26.456 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:26.456 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:26.456 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:26.456 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:26.456 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:26.456 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:26.456 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:27.081 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:10:26 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:10:26] "GET /metrics HTTP/1.1" 200 38531 "" "Prometheus/2.33.4" 2024-06-01T02:10:27.432 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:27.432 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:27.432 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:27.432 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:27.432 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:27.432 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:27.433 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:27.433 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:27.433 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:27.433 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:27.433 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:27.433 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:27.433 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:27.433 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: pgmap v241: 161 pgs: 161 active+clean; 585 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-06-01T02:10:27.433 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:27.433 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:27.433 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:27.433 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:27.433 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:27.433 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:27.433 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:27.434 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:27.434 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:27.434 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:27.434 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:27.434 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:27.434 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:27.434 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:27.434 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:27.434 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:27.434 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:27.434 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:27.434 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:27.434 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:27.434 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:27.434 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:27.435 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:27.435 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:27.435 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:27.435 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:27.435 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:27.435 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:27.435 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:27.435 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:27.435 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:27.435 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:27.435 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:27.435 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:27.435 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:27.435 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:27.435 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:27.435 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:27.436 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:27.436 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:27.436 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:27.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:27.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:27.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:27.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:27.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:27.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:27.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:27.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:27.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:27.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:27.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:27.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:27.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:27.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: pgmap v241: 161 pgs: 161 active+clean; 585 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-06-01T02:10:27.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:27.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:27.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:27.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:27.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:27.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:27.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:27.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:27.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:27.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:27.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:27.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:27.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:27.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:27.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:27.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:27.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:27.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:27.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:27.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:27.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:27.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:27.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:27.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:27.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:27.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:27.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:27.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: pgmap v241: 161 pgs: 161 active+clean; 585 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-06-01T02:10:27.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:27.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:27.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:27.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:27.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:27.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:27.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:27.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:27.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:27.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:27.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:27.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:27.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:27.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:27.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:27.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:27.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:27.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:27.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:27.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:27.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:27.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:27.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:27.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:27.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:27.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:27.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:27.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:27.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:27.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:27.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:27.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:27.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:27.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:27.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:27.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:27.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:27.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:27.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:27.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:27.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:27.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:27.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:27.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:27.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:27.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:27.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:27.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:27.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:27.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:27.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:27.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:27.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:27.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:27.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:27.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:27.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:27.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:27.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:27.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:27.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:27.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:27.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:27.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:27.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:27.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:27.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:27.551 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:28.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:28.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:28.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:28.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:28.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:28.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:28.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:28.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:28.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:28.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:28.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:28.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:28.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:28.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:28.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:28.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:28.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:28.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:28.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:28.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:28.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:28.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:28.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:28.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:28.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:28.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:28.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:28.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:28.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:28.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:28.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:28.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:28.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:28.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:28.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:28.446 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:28.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:28.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:28.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:28.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:28.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:28.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:28.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:28.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:28.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:28.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:28.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:28.447 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:28.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:28.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:28.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:28.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:28.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:28.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:28.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:28.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:28.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:28.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:28.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:28.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:28.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:28.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:28.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:28.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:28.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:28.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:28.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:28.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:28.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:28.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:28.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:28.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:28.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:28.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:28.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:28.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:28.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:28.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:28.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:28.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:28.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:28.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:28.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:28.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:28.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:28.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:28.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:28.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:28.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:28.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:28.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:28.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:28.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:28.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:28.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:28.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:28.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:28.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:28.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:28.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:28.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:28.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:28.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:28.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:28.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:28.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:28.547 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:28.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:28.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:28.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:28.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:28.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:28.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:28.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:28.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:28.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:28.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:28.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:28.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:28.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:28.548 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:28.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:28.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:28.549 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:28.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:28.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:28.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:28.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:28.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:28.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:28.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:28.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:28.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:28.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:28.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:28.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:28.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:28.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:28.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:28.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:28.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:28.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:28.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:28.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:29.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:29.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:29.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:29.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:29.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:29.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:29.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:29.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: pgmap v242: 161 pgs: 161 active+clean; 585 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 785 B/s rd, 0 op/s 2024-06-01T02:10:29.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:29.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:29.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:29.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:29.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:29.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:29.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:29.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:29.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:29.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:29.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:29.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:29.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:29.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:29.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:29.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:29.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:29.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:29.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:29.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:29.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:29.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:29.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:29.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:29.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:29.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:29.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:29.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:29.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:29.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:29.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:29.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:29.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:29.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:29.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:29.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:29.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:29.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:29.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:29.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:29.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:29.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:29.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: pgmap v242: 161 pgs: 161 active+clean; 585 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 785 B/s rd, 0 op/s 2024-06-01T02:10:29.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:29.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:29.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:29.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:29.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:29.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:29.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:29.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:29.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:29.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:29.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:29.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:29.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:29.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:29.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:29.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:29.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:29.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:29.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:29.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:29.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:29.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:29.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:29.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:29.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:29.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:29.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:29.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: pgmap v242: 161 pgs: 161 active+clean; 585 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 785 B/s rd, 0 op/s 2024-06-01T02:10:29.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:29.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:29.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:29.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:29.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:29.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:29.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:29.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:29.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:29.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:29.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:29.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:29.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:29.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:29.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:29.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:29.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:29.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:29.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:29.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:29.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:29.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:29.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:29.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:29.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:29.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:29.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:29.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:29.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:29.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:29.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:29.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:29.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:29.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:29.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:29.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:29.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:29.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:29.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:29.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:29.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:29.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:29.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:29.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:29.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:29.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:29.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:29.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:29.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:29.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:30.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-01T02:10:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-01T02:10:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: Upgrade: osd.5 is safe to restart 2024-06-01T02:10:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:30.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:30.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:30.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:30.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:30.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:30.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:30.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:30.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:30.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:30.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:30.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:30.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:30.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:30.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:30.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:30.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:30.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:30.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:30.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:30.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:30.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:30.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:30.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:30.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:30.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:30.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:30.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:30.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:30.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:30.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:30.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:30.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:30.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-01T02:10:30.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-01T02:10:30.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: Upgrade: osd.5 is safe to restart 2024-06-01T02:10:30.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:30.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:30.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:30.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:30.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:30.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:30.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:30.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:30.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:30.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:30.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:30.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:30.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:30.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:30.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:30.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:30.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:30.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:30.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:30.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:30.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:30.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:30.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:30.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:30.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:30.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:30.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:30.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:30.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:30.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:30.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:30.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:30.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:30.549 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:30.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:30.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-01T02:10:30.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-01T02:10:30.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: Upgrade: osd.5 is safe to restart 2024-06-01T02:10:30.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:30.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:30.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:31.754 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:31 smithi161 ceph-mon[84406]: pgmap v243: 161 pgs: 161 active+clean; 585 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 708 B/s rd, 0 op/s 2024-06-01T02:10:31.754 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:31.754 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:31.754 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:31.754 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:31.754 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:10:31.754 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:32.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:31 smithi116 ceph-mon[89502]: pgmap v243: 161 pgs: 161 active+clean; 585 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 708 B/s rd, 0 op/s 2024-06-01T02:10:32.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:32.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:32.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:32.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:32.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:10:32.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:32.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:31 smithi116 ceph-mon[100606]: pgmap v243: 161 pgs: 161 active+clean; 585 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 708 B/s rd, 0 op/s 2024-06-01T02:10:32.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:32.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:32.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:32.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:32.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:10:32.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:32.828 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-01T02:10:32.828 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:33.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-01T02:10:33.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:33.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-01T02:10:33.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:33.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:33 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:10:33.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:33 smithi161 ceph-mon[84406]: pgmap v244: 161 pgs: 161 active+clean; 585 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 597 B/s rd, 0 op/s 2024-06-01T02:10:33.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:33 smithi161 ceph-mon[84406]: Upgrade: Updating osd.5 2024-06-01T02:10:33.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:33 smithi161 ceph-mon[84406]: Deploying daemon osd.5 on smithi161 2024-06-01T02:10:33.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.16"}]: dispatch 2024-06-01T02:10:33.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.17"}]: dispatch 2024-06-01T02:10:33.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:33.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:33.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:34.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:33 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:10:34.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:33 smithi116 ceph-mon[89502]: pgmap v244: 161 pgs: 161 active+clean; 585 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 597 B/s rd, 0 op/s 2024-06-01T02:10:34.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:33 smithi116 ceph-mon[89502]: Upgrade: Updating osd.5 2024-06-01T02:10:34.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:33 smithi116 ceph-mon[89502]: Deploying daemon osd.5 on smithi161 2024-06-01T02:10:34.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.16"}]: dispatch 2024-06-01T02:10:34.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.17"}]: dispatch 2024-06-01T02:10:34.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:34.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:34.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:34.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:33 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:10:34.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:33 smithi116 ceph-mon[100606]: pgmap v244: 161 pgs: 161 active+clean; 585 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 597 B/s rd, 0 op/s 2024-06-01T02:10:34.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:33 smithi116 ceph-mon[100606]: Upgrade: Updating osd.5 2024-06-01T02:10:34.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:33 smithi116 ceph-mon[100606]: Deploying daemon osd.5 on smithi161 2024-06-01T02:10:34.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.16"}]: dispatch 2024-06-01T02:10:34.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.17"}]: dispatch 2024-06-01T02:10:34.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:34.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:34.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:34.791 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.16"}]': finished 2024-06-01T02:10:34.791 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.17"}]': finished 2024-06-01T02:10:34.791 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:34 smithi161 ceph-mon[84406]: osdmap e171: 8 total, 8 up, 8 in 2024-06-01T02:10:35.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.16"}]': finished 2024-06-01T02:10:35.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.17"}]': finished 2024-06-01T02:10:35.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:34 smithi116 ceph-mon[100606]: osdmap e171: 8 total, 8 up, 8 in 2024-06-01T02:10:35.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.16"}]': finished 2024-06-01T02:10:35.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.17"}]': finished 2024-06-01T02:10:35.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:34 smithi116 ceph-mon[89502]: osdmap e171: 8 total, 8 up, 8 in 2024-06-01T02:10:35.830 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:35 smithi161 ceph-mon[84406]: pgmap v246: 161 pgs: 161 active+clean; 585 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:10:35.831 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:35 smithi161 ceph-mon[84406]: osdmap e172: 8 total, 8 up, 8 in 2024-06-01T02:10:36.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:35 smithi116 ceph-mon[89502]: pgmap v246: 161 pgs: 161 active+clean; 585 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:10:36.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:35 smithi116 ceph-mon[89502]: osdmap e172: 8 total, 8 up, 8 in 2024-06-01T02:10:36.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:35 smithi116 ceph-mon[100606]: pgmap v246: 161 pgs: 161 active+clean; 585 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:10:36.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:35 smithi116 ceph-mon[100606]: osdmap e172: 8 total, 8 up, 8 in 2024-06-01T02:10:36.182 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:35 smithi161 systemd[1]: Stopping Ceph osd.5 for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:10:36.444 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:36 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[108027]: 2024-06-01T02:10:36.186+0000 7fbb16caa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-01T02:10:36.444 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:36 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[108027]: 2024-06-01T02:10:36.186+0000 7fbb16caa700 -1 osd.5 173 *** Got signal Terminated *** 2024-06-01T02:10:36.444 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:36 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[108027]: 2024-06-01T02:10:36.186+0000 7fbb16caa700 -1 osd.5 173 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-01T02:10:36.875 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:36 smithi161 ceph-mon[84406]: osdmap e173: 8 total, 8 up, 8 in 2024-06-01T02:10:36.875 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:36 smithi161 ceph-mon[84406]: osd.5 marked itself down and dead 2024-06-01T02:10:36.876 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:36 smithi161 ceph-mon[84406]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:10:36.876 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:36 smithi161 ceph-mon[84406]: osdmap e174: 8 total, 7 up, 8 in 2024-06-01T02:10:36.876 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:36 smithi161 ceph-mon[84406]: osd.5 marked itself dead as of e174 2024-06-01T02:10:36.876 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:10:36 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:10:36] "GET /metrics HTTP/1.1" 200 38532 "" "Prometheus/2.33.4" 2024-06-01T02:10:36.905 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:36 smithi116 ceph-mon[100606]: osdmap e173: 8 total, 8 up, 8 in 2024-06-01T02:10:36.905 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:36 smithi116 ceph-mon[100606]: osd.5 marked itself down and dead 2024-06-01T02:10:36.905 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:36 smithi116 ceph-mon[100606]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:10:36.905 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:36 smithi116 ceph-mon[100606]: osdmap e174: 8 total, 7 up, 8 in 2024-06-01T02:10:36.906 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:36 smithi116 ceph-mon[100606]: osd.5 marked itself dead as of e174 2024-06-01T02:10:36.906 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:36 smithi116 ceph-mon[89502]: osdmap e173: 8 total, 8 up, 8 in 2024-06-01T02:10:36.906 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:36 smithi116 ceph-mon[89502]: osd.5 marked itself down and dead 2024-06-01T02:10:36.906 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:36 smithi116 ceph-mon[89502]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:10:36.906 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:36 smithi116 ceph-mon[89502]: osdmap e174: 8 total, 7 up, 8 in 2024-06-01T02:10:36.906 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:36 smithi116 ceph-mon[89502]: osd.5 marked itself dead as of e174 2024-06-01T02:10:36.906 INFO:teuthology.orchestra.run.smithi116.stdout:true 2024-06-01T02:10:37.191 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:36 smithi161 podman[114190]: 2024-06-01 02:10:36.880578476 +0000 UTC m=+0.792236276 container died 596f1b7846a2f9d7bcc47103ca40d1ec3f32b75da3fc0c254df90d2ea5635179 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-06-01T02:10:37.444 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:37 smithi161 podman[114190]: 2024-06-01 02:10:37.369924607 +0000 UTC m=+1.281582381 container cleanup 596f1b7846a2f9d7bcc47103ca40d1ec3f32b75da3fc0c254df90d2ea5635179 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True) 2024-06-01T02:10:37.444 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:37 smithi161 bash[114190]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5 2024-06-01T02:10:37.566 INFO:teuthology.orchestra.run.smithi116.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-01T02:10:37.566 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi116 smithi116 running 4s ago 19m - - 2024-06-01T02:10:37.566 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi161 smithi161 running 6s ago 19m - - 2024-06-01T02:10:37.566 INFO:teuthology.orchestra.run.smithi116.stdout:alertmanager.a smithi116 *:9093,9094 running (8m) 4s ago 13m 22.0M - 0.23.0 ba2b418f427c e0c297162ded 2024-06-01T02:10:37.566 INFO:teuthology.orchestra.run.smithi116.stdout:grafana.a smithi161 *:3000 running (8m) 6s ago 13m 52.9M - 8.3.5 dad864ee21e9 00b0a78ee88e 2024-06-01T02:10:37.567 INFO:teuthology.orchestra.run.smithi116.stdout:iscsi.foo.smithi116.gjuxzb smithi116 running (7m) 4s ago 12m 62.9M - 3.5 e1d6a67b021e 0e191fe3e3bb 2024-06-01T02:10:37.567 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.x smithi161 *:8443,9283,8765 running (7m) 6s ago 18m 530M - 19.0.0-2946-g3d89accf b045ac545f82 2d7fbb85e5f7 2024-06-01T02:10:37.567 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.y smithi116 *:8443,9283,8765 running (6m) 4s ago 20m 455M - 19.0.0-2946-g3d89accf b045ac545f82 bc9627c27a57 2024-06-01T02:10:37.567 INFO:teuthology.orchestra.run.smithi116.stdout:mon.a smithi116 running (5m) 4s ago 20m 68.4M 2048M 19.0.0-2946-g3d89accf b045ac545f82 b5e9b6f873ff 2024-06-01T02:10:37.567 INFO:teuthology.orchestra.run.smithi116.stdout:mon.b smithi161 running (4m) 6s ago 18m 48.5M 2048M 19.0.0-2946-g3d89accf b045ac545f82 aaa08c03cfc8 2024-06-01T02:10:37.567 INFO:teuthology.orchestra.run.smithi116.stdout:mon.c smithi116 running (5m) 4s ago 18m 51.4M 2048M 19.0.0-2946-g3d89accf b045ac545f82 bd0fb0e603fa 2024-06-01T02:10:37.567 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.a smithi116 *:9100 running (8m) 4s ago 14m 17.1M - 1.3.1 1dbe0e931976 e347748d17bb 2024-06-01T02:10:37.567 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.b smithi161 *:9100 running (8m) 6s ago 14m 20.8M - 1.3.1 1dbe0e931976 6e66182f5a1a 2024-06-01T02:10:37.567 INFO:teuthology.orchestra.run.smithi116.stdout:osd.0 smithi116 running (3m) 4s ago 18m 50.9M 4096M 19.0.0-2946-g3d89accf b045ac545f82 b4313d121b9d 2024-06-01T02:10:37.567 INFO:teuthology.orchestra.run.smithi116.stdout:osd.1 smithi116 running (2m) 4s ago 17m 54.4M 4096M 19.0.0-2946-g3d89accf b045ac545f82 dccdbeac5aae 2024-06-01T02:10:37.567 INFO:teuthology.orchestra.run.smithi116.stdout:osd.2 smithi116 running (2m) 4s ago 17m 48.1M 4096M 19.0.0-2946-g3d89accf b045ac545f82 1ad95efda404 2024-06-01T02:10:37.567 INFO:teuthology.orchestra.run.smithi116.stdout:osd.3 smithi116 running (115s) 4s ago 16m 51.1M 4096M 19.0.0-2946-g3d89accf b045ac545f82 bd00ad2c2556 2024-06-01T02:10:37.567 INFO:teuthology.orchestra.run.smithi116.stdout:osd.4 smithi161 running (64s) 6s ago 16m 49.8M 4096M 19.0.0-2946-g3d89accf b045ac545f82 0084b9edf7a0 2024-06-01T02:10:37.567 INFO:teuthology.orchestra.run.smithi116.stdout:osd.5 smithi161 running (24s) 6s ago 16m 46.3M 4096M 19.0.0-2946-g3d89accf b045ac545f82 596f1b7846a2 2024-06-01T02:10:37.568 INFO:teuthology.orchestra.run.smithi116.stdout:osd.6 smithi161 running (15m) 6s ago 15m 61.7M 4096M 17.2.0 e1d6a67b021e 50abeb0551e7 2024-06-01T02:10:37.568 INFO:teuthology.orchestra.run.smithi116.stdout:osd.7 smithi161 running (15m) 6s ago 15m 66.1M 4096M 17.2.0 e1d6a67b021e 6004e6983735 2024-06-01T02:10:37.568 INFO:teuthology.orchestra.run.smithi116.stdout:prometheus.a smithi161 *:9095 running (7m) 6s ago 14m 76.9M - 2.33.4 514e6a882f6e e977f6ca1d73 2024-06-01T02:10:37.568 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi116.vzjrxj smithi116 *:8000 running (12m) 4s ago 12m 77.6M - 17.2.0 e1d6a67b021e df4fe3c1dbc3 2024-06-01T02:10:37.568 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi161.ashqxz smithi161 *:8000 running (12m) 6s ago 12m 78.2M - 17.2.0 e1d6a67b021e 3eecc1f3e2d3 2024-06-01T02:10:37.568 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi116.hokbmf smithi116 *:80 running (12m) 4s ago 12m 77.9M - 17.2.0 e1d6a67b021e 10950ed75dbd 2024-06-01T02:10:37.568 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi161.axjlzw smithi161 *:80 running (12m) 6s ago 12m 77.3M - 17.2.0 e1d6a67b021e 0b5d5ce47772 2024-06-01T02:10:37.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:37 smithi161 ceph-mon[84406]: pgmap v249: 161 pgs: 161 active+clean; 585 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:10:37.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:37 smithi161 ceph-mon[84406]: from='client.64471 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:10:37.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:37 smithi161 ceph-mon[84406]: osdmap e175: 8 total, 7 up, 8 in 2024-06-01T02:10:37.944 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:37 smithi161 podman[114207]: 2024-06-01 02:10:37.655447512 +0000 UTC m=+0.769577252 container remove 596f1b7846a2f9d7bcc47103ca40d1ec3f32b75da3fc0c254df90d2ea5635179 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc) 2024-06-01T02:10:37.957 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:10:37.957 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": { 2024-06-01T02:10:37.957 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 3 2024-06-01T02:10:37.957 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:10:37.957 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": { 2024-06-01T02:10:37.958 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-01T02:10:37.958 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:10:37.958 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": { 2024-06-01T02:10:37.958 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 2, 2024-06-01T02:10:37.958 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 5 2024-06-01T02:10:37.958 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:10:37.958 INFO:teuthology.orchestra.run.smithi116.stdout: "rgw": { 2024-06-01T02:10:37.958 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-06-01T02:10:37.958 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:10:37.958 INFO:teuthology.orchestra.run.smithi116.stdout: "overall": { 2024-06-01T02:10:37.958 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 6, 2024-06-01T02:10:37.958 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 10 2024-06-01T02:10:37.959 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T02:10:37.959 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:10:37.959 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:37 smithi116 ceph-mon[89502]: pgmap v249: 161 pgs: 161 active+clean; 585 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:10:37.959 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:37 smithi116 ceph-mon[89502]: from='client.64471 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:10:37.959 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:37 smithi116 ceph-mon[89502]: osdmap e175: 8 total, 7 up, 8 in 2024-06-01T02:10:37.959 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:37 smithi116 ceph-mon[100606]: pgmap v249: 161 pgs: 161 active+clean; 585 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:10:37.959 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:37 smithi116 ceph-mon[100606]: from='client.64471 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:10:37.959 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:37 smithi116 ceph-mon[100606]: osdmap e175: 8 total, 7 up, 8 in 2024-06-01T02:10:38.262 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:37 smithi161 podman[114296]: 2024-06-01 02:10:37.98972792 +0000 UTC m=+0.114086943 container create 9354f21ca54d1be6d49a9f8c16187881a88b40d92e75843c37cf8e2e92694427 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-06-01T02:10:38.262 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:37 smithi161 podman[114296]: 2024-06-01 02:10:37.893050383 +0000 UTC m=+0.017409437 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:10:38.262 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:38 smithi161 podman[114296]: 2024-06-01 02:10:38.192709302 +0000 UTC m=+0.317068336 container init 9354f21ca54d1be6d49a9f8c16187881a88b40d92e75843c37cf8e2e92694427 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2) 2024-06-01T02:10:38.263 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:38 smithi161 podman[114296]: 2024-06-01 02:10:38.196235037 +0000 UTC m=+0.320594080 container start 9354f21ca54d1be6d49a9f8c16187881a88b40d92e75843c37cf8e2e92694427 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528) 2024-06-01T02:10:38.263 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:38 smithi161 podman[114296]: 2024-06-01 02:10:38.267253623 +0000 UTC m=+0.391612661 container attach 9354f21ca54d1be6d49a9f8c16187881a88b40d92e75843c37cf8e2e92694427 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-06-01T02:10:38.286 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:10:38.286 INFO:teuthology.orchestra.run.smithi116.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-01T02:10:38.286 INFO:teuthology.orchestra.run.smithi116.stdout: "in_progress": true, 2024-06-01T02:10:38.286 INFO:teuthology.orchestra.run.smithi116.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-01T02:10:38.286 INFO:teuthology.orchestra.run.smithi116.stdout: "services_complete": [ 2024-06-01T02:10:38.286 INFO:teuthology.orchestra.run.smithi116.stdout: "mon", 2024-06-01T02:10:38.286 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr" 2024-06-01T02:10:38.286 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-06-01T02:10:38.286 INFO:teuthology.orchestra.run.smithi116.stdout: "progress": "11/23 daemons upgraded", 2024-06-01T02:10:38.286 INFO:teuthology.orchestra.run.smithi116.stdout: "message": "Currently upgrading osd daemons", 2024-06-01T02:10:38.287 INFO:teuthology.orchestra.run.smithi116.stdout: "is_paused": false 2024-06-01T02:10:38.287 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:10:38.602 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:38 smithi161 podman[114296]: 2024-06-01 02:10:38.505942077 +0000 UTC m=+0.630301108 container died 9354f21ca54d1be6d49a9f8c16187881a88b40d92e75843c37cf8e2e92694427 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-deactivate, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-01T02:10:38.680 INFO:teuthology.orchestra.run.smithi116.stdout:HEALTH_WARN 1 osds down; Reduced data availability: 4 pgs inactive, 3 pgs peering; Degraded data redundancy: 37/726 objects degraded (5.096%), 4 pgs degraded 2024-06-01T02:10:38.681 INFO:teuthology.orchestra.run.smithi116.stdout:[WRN] OSD_DOWN: 1 osds down 2024-06-01T02:10:38.681 INFO:teuthology.orchestra.run.smithi116.stdout: osd.5 (root=default,host=smithi161) is down 2024-06-01T02:10:38.681 INFO:teuthology.orchestra.run.smithi116.stdout:[WRN] PG_AVAILABILITY: Reduced data availability: 4 pgs inactive, 3 pgs peering 2024-06-01T02:10:38.681 INFO:teuthology.orchestra.run.smithi116.stdout: pg 4.8 is stuck peering for 13m, current state peering, last acting [7,6] 2024-06-01T02:10:38.681 INFO:teuthology.orchestra.run.smithi116.stdout: pg 4.12 is stuck peering for 2m, current state peering, last acting [6,2] 2024-06-01T02:10:38.681 INFO:teuthology.orchestra.run.smithi116.stdout: pg 5.16 is stuck inactive for 109s, current state activating+undersized, last acting [3,1] 2024-06-01T02:10:38.681 INFO:teuthology.orchestra.run.smithi116.stdout: pg 6.a is stuck peering for 3m, current state peering, last acting [6,0] 2024-06-01T02:10:38.681 INFO:teuthology.orchestra.run.smithi116.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 37/726 objects degraded (5.096%), 4 pgs degraded 2024-06-01T02:10:38.681 INFO:teuthology.orchestra.run.smithi116.stdout: pg 2.18 is active+undersized+degraded, acting [4,1] 2024-06-01T02:10:38.681 INFO:teuthology.orchestra.run.smithi116.stdout: pg 3.1c is active+undersized+degraded, acting [4,1] 2024-06-01T02:10:38.681 INFO:teuthology.orchestra.run.smithi116.stdout: pg 3.1d is active+undersized+degraded, acting [4,6] 2024-06-01T02:10:38.681 INFO:teuthology.orchestra.run.smithi116.stdout: pg 6.1a is active+undersized+degraded, acting [4,1] 2024-06-01T02:10:38.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:38 smithi161 ceph-mon[84406]: from='client.64477 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:10:38.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:38 smithi161 ceph-mon[84406]: from='client.64483 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:10:38.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:38 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/698641650' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:38.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:38 smithi161 ceph-mon[84406]: osdmap e176: 8 total, 7 up, 8 in 2024-06-01T02:10:39.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:38 smithi116 ceph-mon[89502]: from='client.64477 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:10:39.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:38 smithi116 ceph-mon[89502]: from='client.64483 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:10:39.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:38 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/698641650' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:39.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:38 smithi116 ceph-mon[89502]: osdmap e176: 8 total, 7 up, 8 in 2024-06-01T02:10:39.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:38 smithi116 ceph-mon[100606]: from='client.64477 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:10:39.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:38 smithi116 ceph-mon[100606]: from='client.64483 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:10:39.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:38 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/698641650' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:39.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:38 smithi116 ceph-mon[100606]: osdmap e176: 8 total, 7 up, 8 in 2024-06-01T02:10:39.326 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:39 smithi161 podman[114322]: 2024-06-01 02:10:39.063277533 +0000 UTC m=+0.550068099 container remove 9354f21ca54d1be6d49a9f8c16187881a88b40d92e75843c37cf8e2e92694427 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-06-01T02:10:39.326 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:39 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.5.service: Deactivated successfully. 2024-06-01T02:10:39.326 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:39 smithi161 systemd[1]: Stopped Ceph osd.5 for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:10:39.326 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:39 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.5.service: Consumed 1.935s CPU time. 2024-06-01T02:10:39.326 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:39 smithi161 systemd[1]: Starting Ceph osd.5 for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:10:39.603 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:39 smithi161 podman[114424]: 2024-06-01 02:10:39.434516258 +0000 UTC m=+0.018256447 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:10:39.603 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:39 smithi161 podman[114424]: 2024-06-01 02:10:39.56435593 +0000 UTC m=+0.148096113 container create e001c7ef322d2060906e154cd5cc3cf774168b9c3ba79ab7d69151d796bfc7ca (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-activate, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-01T02:10:39.879 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:39 smithi161 ceph-mon[84406]: pgmap v252: 161 pgs: 9 activating+undersized, 4 active+undersized+degraded, 31 peering, 8 stale+active+clean, 5 active+undersized, 104 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 37/726 objects degraded (5.096%) 2024-06-01T02:10:39.879 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:39 smithi161 ceph-mon[84406]: from='client.64495 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:10:39.879 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:39 smithi161 ceph-mon[84406]: Health check failed: Reduced data availability: 4 pgs inactive, 3 pgs peering (PG_AVAILABILITY) 2024-06-01T02:10:39.879 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:39 smithi161 ceph-mon[84406]: Health check failed: Degraded data redundancy: 37/726 objects degraded (5.096%), 4 pgs degraded (PG_DEGRADED) 2024-06-01T02:10:39.879 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:39 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/4197436884' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:10:39.879 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:39 smithi161 podman[114424]: 2024-06-01 02:10:39.8001967 +0000 UTC m=+0.383936888 container init e001c7ef322d2060906e154cd5cc3cf774168b9c3ba79ab7d69151d796bfc7ca (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-activate, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-06-01T02:10:39.879 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:39 smithi161 podman[114424]: 2024-06-01 02:10:39.803791077 +0000 UTC m=+0.387531266 container start e001c7ef322d2060906e154cd5cc3cf774168b9c3ba79ab7d69151d796bfc7ca (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-activate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240528, io.buildah.version=1.33.5) 2024-06-01T02:10:40.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:39 smithi116 ceph-mon[89502]: pgmap v252: 161 pgs: 9 activating+undersized, 4 active+undersized+degraded, 31 peering, 8 stale+active+clean, 5 active+undersized, 104 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 37/726 objects degraded (5.096%) 2024-06-01T02:10:40.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:39 smithi116 ceph-mon[89502]: from='client.64495 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:10:40.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:39 smithi116 ceph-mon[89502]: Health check failed: Reduced data availability: 4 pgs inactive, 3 pgs peering (PG_AVAILABILITY) 2024-06-01T02:10:40.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:39 smithi116 ceph-mon[89502]: Health check failed: Degraded data redundancy: 37/726 objects degraded (5.096%), 4 pgs degraded (PG_DEGRADED) 2024-06-01T02:10:40.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:39 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/4197436884' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:10:40.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:39 smithi116 ceph-mon[100606]: pgmap v252: 161 pgs: 9 activating+undersized, 4 active+undersized+degraded, 31 peering, 8 stale+active+clean, 5 active+undersized, 104 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 37/726 objects degraded (5.096%) 2024-06-01T02:10:40.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:39 smithi116 ceph-mon[100606]: from='client.64495 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:10:40.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:39 smithi116 ceph-mon[100606]: Health check failed: Reduced data availability: 4 pgs inactive, 3 pgs peering (PG_AVAILABILITY) 2024-06-01T02:10:40.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:39 smithi116 ceph-mon[100606]: Health check failed: Degraded data redundancy: 37/726 objects degraded (5.096%), 4 pgs degraded (PG_DEGRADED) 2024-06-01T02:10:40.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:39 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/4197436884' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:10:40.194 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:39 smithi161 podman[114424]: 2024-06-01 02:10:39.883457323 +0000 UTC m=+0.467197523 container attach e001c7ef322d2060906e154cd5cc3cf774168b9c3ba79ab7d69151d796bfc7ca (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-activate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-01T02:10:40.583 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:40 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-activate[114446]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-06-01T02:10:40.583 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:40 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-activate[114446]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-06-01T02:10:40.583 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:40 smithi161 bash[114424]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-06-01T02:10:40.583 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:40 smithi161 bash[114424]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-06-01T02:10:40.583 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:40 smithi161 bash[114424]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-06-01T02:10:40.583 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:40 smithi161 bash[114424]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-06-01T02:10:40.583 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:40 smithi161 bash[114424]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-5/block 2024-06-01T02:10:40.583 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:40 smithi161 bash[114424]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-06-01T02:10:40.584 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:40 smithi161 bash[114424]: --> ceph-volume raw activate successful for osd ID: 5 2024-06-01T02:10:40.584 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:40 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-activate[114446]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-06-01T02:10:40.584 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:40 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-activate[114446]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-06-01T02:10:40.584 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:40 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-activate[114446]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-5/block 2024-06-01T02:10:40.584 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:40 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-activate[114446]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-06-01T02:10:40.584 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:40 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-activate[114446]: --> ceph-volume raw activate successful for osd ID: 5 2024-06-01T02:10:40.584 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:40 smithi161 podman[114424]: 2024-06-01 02:10:40.285571222 +0000 UTC m=+0.869311411 container died e001c7ef322d2060906e154cd5cc3cf774168b9c3ba79ab7d69151d796bfc7ca (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-activate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-06-01T02:10:41.310 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:40 smithi161 podman[114576]: 2024-06-01 02:10:40.987655022 +0000 UTC m=+0.693456386 container remove e001c7ef322d2060906e154cd5cc3cf774168b9c3ba79ab7d69151d796bfc7ca (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-activate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-06-01T02:10:41.310 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:41 smithi161 podman[114636]: 2024-06-01 02:10:41.10872336 +0000 UTC m=+0.017345832 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:10:41.310 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:41 smithi161 podman[114636]: 2024-06-01 02:10:41.213631258 +0000 UTC m=+0.122253729 container create b3849cdeb707262a58ce5c90479519664aa4080aadb8283e99ca0277b57f68eb (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, CEPH_POINT_RELEASE=) 2024-06-01T02:10:41.714 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:41 smithi116 ceph-mon[89502]: pgmap v254: 161 pgs: 9 activating+undersized, 4 active+undersized+degraded, 31 peering, 8 stale+active+clean, 5 active+undersized, 104 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 441 B/s rd, 0 op/s; 37/726 objects degraded (5.096%) 2024-06-01T02:10:41.714 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:41.714 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:41.714 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:41.714 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:41.715 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:41 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:41.715 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:41 smithi116 ceph-mon[100606]: pgmap v254: 161 pgs: 9 activating+undersized, 4 active+undersized+degraded, 31 peering, 8 stale+active+clean, 5 active+undersized, 104 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 441 B/s rd, 0 op/s; 37/726 objects degraded (5.096%) 2024-06-01T02:10:41.715 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:41.715 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:41.715 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:41.715 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:41.715 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:41 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:41.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:41 smithi161 ceph-mon[84406]: pgmap v254: 161 pgs: 9 activating+undersized, 4 active+undersized+degraded, 31 peering, 8 stale+active+clean, 5 active+undersized, 104 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 441 B/s rd, 0 op/s; 37/726 objects degraded (5.096%) 2024-06-01T02:10:41.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:41.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:41.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:41.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:41.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:41 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:41.944 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:41 smithi161 podman[114636]: 2024-06-01 02:10:41.449576576 +0000 UTC m=+0.358199059 container init b3849cdeb707262a58ce5c90479519664aa4080aadb8283e99ca0277b57f68eb (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-06-01T02:10:41.944 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:41 smithi161 podman[114636]: 2024-06-01 02:10:41.453211996 +0000 UTC m=+0.361834469 container start b3849cdeb707262a58ce5c90479519664aa4080aadb8283e99ca0277b57f68eb (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-01T02:10:41.944 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:41 smithi161 ceph-osd[114655]: -- 172.21.15.161:0/2872976778 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56436a613680 con 0x56436a5dd400 2024-06-01T02:10:41.945 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:41 smithi161 bash[114636]: b3849cdeb707262a58ce5c90479519664aa4080aadb8283e99ca0277b57f68eb 2024-06-01T02:10:41.945 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:41 smithi161 systemd[1]: Started Ceph osd.5 for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:10:42.443 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:41 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:10:41.989+0000 7fecb95787c0 -1 Falling back to public interface 2024-06-01T02:10:42.736 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:42.736 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:42.737 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:42.737 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:42.737 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:42.737 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:42.737 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:42.737 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:42.737 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:42.737 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:42.737 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:42.737 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:42.737 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:42.737 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:42.737 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:42.737 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:42.738 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:42.738 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:42.738 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:42.738 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:42.738 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:42.738 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:42.738 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:42.738 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:42.738 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:42.738 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:42.738 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:42.738 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:42.739 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:42.739 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:42.739 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:42.739 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:42.739 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:42.739 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:42.739 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:42.739 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:42.739 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:42.739 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:42.739 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:42.739 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:42.740 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:42.740 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:42.740 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:42.740 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:42.740 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:42.740 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:42.740 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:42.740 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:42.740 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:42.740 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:42.740 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:42.740 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:42.740 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:42.740 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:42.741 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:42.741 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:42.741 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:42.741 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:42.741 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:42.741 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:42.741 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:42.741 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:42.741 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:42.741 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:42.741 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:42.741 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:42.742 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:42.742 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:42.742 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:42.742 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:42.742 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:42.742 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:42.742 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:42.742 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:42.742 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:42.742 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:42.899 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:42.899 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:42.899 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:42.900 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:42.900 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:42.900 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:42.900 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:42.900 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:42.900 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:42.900 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:42.900 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:42.900 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:42.900 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:42.900 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:42.900 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:42.900 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:42.900 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:42.901 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:42.901 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:42.901 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:42.901 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:42.901 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:42.901 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:42.901 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:42.901 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:42.901 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:42.901 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:42.901 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:42.901 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:42.901 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:42.901 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:42.901 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:42.901 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:42.902 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:42.902 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:42.902 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:42.902 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:42.902 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:43.703 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:43.703 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:10:43.703 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:43.703 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: pgmap v255: 161 pgs: 104 active+clean, 16 active+undersized, 6 stale+active+clean, 13 peering, 12 active+undersized+degraded, 1 active+clean+remapped, 9 activating+undersized; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 341 B/s rd, 0 op/s; 53/726 objects degraded (7.300%); 6/726 objects misplaced (0.826%) 2024-06-01T02:10:43.704 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:43.704 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:43.704 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:43.704 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:43.704 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:43.704 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:43.704 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:43.704 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:43.704 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:43.704 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:43.705 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:43.705 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:43.705 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:43.705 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:43.705 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:43.705 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:43.705 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:43.705 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:43.705 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:43.705 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:43.705 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:43.705 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:43.705 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:43.706 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:43.706 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:43.706 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:10:43.706 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:43.706 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: pgmap v255: 161 pgs: 104 active+clean, 16 active+undersized, 6 stale+active+clean, 13 peering, 12 active+undersized+degraded, 1 active+clean+remapped, 9 activating+undersized; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 341 B/s rd, 0 op/s; 53/726 objects degraded (7.300%); 6/726 objects misplaced (0.826%) 2024-06-01T02:10:43.706 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:43.706 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:43.707 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:43.707 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:43.707 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:43.707 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:43.707 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:43.707 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:43.707 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:43.707 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:43.707 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:43.707 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:43.707 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:43.707 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:43.707 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:43.707 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:43.708 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:43.708 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:43.708 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:43.708 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:43.708 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:43.708 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:43.708 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:43.708 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:43.708 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:43.708 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:43.708 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:43.708 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:43.708 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:43.709 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:43.709 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:43.709 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:43.709 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:43.709 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:43.709 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:43.709 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:43.709 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:43.709 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:43.709 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:43.710 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:43.710 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:43.710 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:43.710 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:43.710 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:43.710 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:43.710 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:43.710 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:43.710 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:43.711 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:43.711 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:43.711 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:43.711 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:43.711 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:43.711 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:43.711 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:43.711 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:43.711 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:43.711 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:43.711 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:43.711 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:43.711 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:43.711 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:43.712 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:43.712 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:43.712 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:43.712 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:43.712 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:43.712 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:43 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:43.864 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:43.864 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:10:43.864 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:43.864 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: pgmap v255: 161 pgs: 104 active+clean, 16 active+undersized, 6 stale+active+clean, 13 peering, 12 active+undersized+degraded, 1 active+clean+remapped, 9 activating+undersized; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 341 B/s rd, 0 op/s; 53/726 objects degraded (7.300%); 6/726 objects misplaced (0.826%) 2024-06-01T02:10:43.864 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:43.864 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:43.864 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:43.864 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:43.865 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:43.865 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:43.865 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:43.865 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:43.865 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:43.865 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:43.865 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:43.865 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:43.865 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:43.865 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:43.865 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:43.865 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:43.865 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:43.866 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:43.866 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:43.866 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:43.866 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:43.866 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:43.866 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:43.866 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:43.866 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:43.866 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:43.866 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:43.866 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:43.866 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:43.867 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:43.867 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:43.867 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:43.867 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:43.867 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:43.867 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:43.867 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:43.867 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:43.867 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:43.867 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:43.867 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:43.867 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:43.867 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:43.867 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:43.868 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:43.868 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:43.868 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:43 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:44.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:44.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:44.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:44.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:44.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:44.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:44.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:44.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:44.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:44.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:44.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:44.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:44.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:44.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:44.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:44.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:44.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:44.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:44.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:44.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:44.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:44.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:44.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:44.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:44.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:44.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:44.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:44.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:44.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:44.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:44.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:44.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:44.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:44.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:44.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:44.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:44.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:44.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:44.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:44.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:44.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:44.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:44.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:44.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:44.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:44.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:44.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:44.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:44.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:44.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:44.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:44.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:44.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:44.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:44.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:44.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:44.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:44.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:44.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:44.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:44.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:44.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:44.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:44.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:44.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:44.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:44.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:44.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:44.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:44.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:44.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:44.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:44.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:44.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:44.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:44.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:44.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:44.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:44.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:44.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:44.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:44.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:44.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:44.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:44.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:44.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:44 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:44.944 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:10:44.748+0000 7fecb95787c0 -1 osd.5 174 log_to_monitors true 2024-06-01T02:10:44.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:44.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:44.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:44.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:44.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:44.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:44.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:44.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:44.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:44.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:44.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:44.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:44.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:44.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:44.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:44.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:44.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:44.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:44.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:44 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:45.944 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:10:45.680+0000 7fecaa9cf700 -1 osd.5 174 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-01T02:10:45.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:45.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:45.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: pgmap v256: 161 pgs: 104 active+clean, 34 active+undersized, 22 active+undersized+degraded, 1 active+clean+remapped; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 908 B/s rd, 0 op/s; 75/726 objects degraded (10.331%); 6/726 objects misplaced (0.826%) 2024-06-01T02:10:45.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:45.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:45.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='osd.5 [v2:172.21.15.161:6808/3624694504,v1:172.21.15.161:6809/3624694504]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-01T02:10:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-01T02:10:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:45.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:45.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:45.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:45.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:45.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:45.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:45.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:45.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:45.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:45.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: Health check update: Degraded data redundancy: 75/726 objects degraded (10.331%), 22 pgs degraded (PG_DEGRADED) 2024-06-01T02:10:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-06-01T02:10:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:45.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:45.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:45.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:45.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:45.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:45.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:45.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:45.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:45.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:45.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:45.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:45.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:45.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:45.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:45.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:45.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:45.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:45.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:46.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:46.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:46.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: pgmap v256: 161 pgs: 104 active+clean, 34 active+undersized, 22 active+undersized+degraded, 1 active+clean+remapped; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 908 B/s rd, 0 op/s; 75/726 objects degraded (10.331%); 6/726 objects misplaced (0.826%) 2024-06-01T02:10:46.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:46.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:46.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:46.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:46.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:46.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:46.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:46.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:46.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='osd.5 [v2:172.21.15.161:6808/3624694504,v1:172.21.15.161:6809/3624694504]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-01T02:10:46.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-01T02:10:46.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:46.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:46.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:46.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:46.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:46.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:46.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:46.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:46.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:46.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:46.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:46.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:46.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:46.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:46.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:46.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:46.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: pgmap v256: 161 pgs: 104 active+clean, 34 active+undersized, 22 active+undersized+degraded, 1 active+clean+remapped; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 908 B/s rd, 0 op/s; 75/726 objects degraded (10.331%); 6/726 objects misplaced (0.826%) 2024-06-01T02:10:46.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:46.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:46.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:46.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:46.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:46.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:46.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:46.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:46.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='osd.5 [v2:172.21.15.161:6808/3624694504,v1:172.21.15.161:6809/3624694504]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-01T02:10:46.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-01T02:10:46.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:46.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:46.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:46.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:46.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:46.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:46.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:46.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:46.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:46.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:46.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:46.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:46.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:46.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:46.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:46.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:46.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:46.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:46.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:46.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:46.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:46.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:46.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: Health check update: Degraded data redundancy: 75/726 objects degraded (10.331%), 22 pgs degraded (PG_DEGRADED) 2024-06-01T02:10:46.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-06-01T02:10:46.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:46.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:46.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:46.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:46.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:46.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:46.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:46.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:46.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:46.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:46.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:46.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:46.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:46.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:46.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:46.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:46.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:46.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:46.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:46.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:46.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:46.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:46.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:46.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:46.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:46.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:46.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:46.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:46.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: Health check update: Degraded data redundancy: 75/726 objects degraded (10.331%), 22 pgs degraded (PG_DEGRADED) 2024-06-01T02:10:46.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-06-01T02:10:46.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:46.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:46.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:46.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:46.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:46.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:46.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:46.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:46.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:46.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:46.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:46.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:46.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:46.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:46.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:46.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:46.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:46.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:46.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:46.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:46.944 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:10:46] "GET /metrics HTTP/1.1" 200 38395 "" "Prometheus/2.33.4" 2024-06-01T02:10:46.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:46.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:46.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:46.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:46.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:46.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:46.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-01T02:10:46.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='osd.5 [v2:172.21.15.161:6808/3624694504,v1:172.21.15.161:6809/3624694504]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:10:46.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: osdmap e177: 8 total, 7 up, 8 in 2024-06-01T02:10:46.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:10:46.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:46.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:46.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:46.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:46.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:46.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:46.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:46.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:46.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:46.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:46.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:46.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:46.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:46.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:10:46.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:46.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:46.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:46.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:46.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:46.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:46.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:46.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:46.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:46.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:46.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:46.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:46.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:46.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:46.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:46.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:46.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:46.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:46.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:46.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:46.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:46.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:46.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:46.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:46.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:46.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:46.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:46.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:46.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:46.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:46.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:46.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:46.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:46.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:46.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:46.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:47.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:47.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:47.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:47.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:47.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:47.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:47.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-01T02:10:47.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='osd.5 [v2:172.21.15.161:6808/3624694504,v1:172.21.15.161:6809/3624694504]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:10:47.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: osdmap e177: 8 total, 7 up, 8 in 2024-06-01T02:10:47.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:10:47.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:47.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:47.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:47.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:47.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:47.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:47.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:47.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:47.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:47.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:47.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:47.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:47.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:47.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:10:47.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:47.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:47.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:47.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:47.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:47.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:47.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:47.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:47.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:47.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:47.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:47.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-01T02:10:47.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='osd.5 [v2:172.21.15.161:6808/3624694504,v1:172.21.15.161:6809/3624694504]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:10:47.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: osdmap e177: 8 total, 7 up, 8 in 2024-06-01T02:10:47.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:10:47.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:47.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:47.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:47.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:47.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:47.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:47.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:47.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:47.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:47.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:47.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:47.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:47.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:47.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:10:47.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:47.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:47.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:47.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:47.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:47.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:47.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:47.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:47.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:47.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:47.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:47.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:47.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:47.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:47.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:47.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:47.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:47.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:47.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:47.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:47.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:47.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:47.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:47.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:47.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:47.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:47.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:47.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:47.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:47.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:47.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:47.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:47.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:47.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:47.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:47.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:47.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:47.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:47.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:47.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:47.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:47.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:47.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:47.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:47.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:47.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:47.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:47.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:47.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:47.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:47.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:47.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:47.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:47.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:47.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:47.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:47.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:47.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:47.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:47.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:47.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:47.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:47.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:47.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:47.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:47.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:47.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:47.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:47.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:47.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: pgmap v258: 161 pgs: 104 active+clean, 34 active+undersized, 22 active+undersized+degraded, 1 active+clean+remapped; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 75/726 objects degraded (10.331%); 6/726 objects misplaced (0.826%) 2024-06-01T02:10:47.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:47.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:47.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:47.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:47.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:10:47.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: osd.5 [v2:172.21.15.161:6808/3624694504,v1:172.21.15.161:6809/3624694504] boot 2024-06-01T02:10:47.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: osdmap e178: 8 total, 8 up, 8 in 2024-06-01T02:10:47.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T02:10:47.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:47.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:47.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:47.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:47.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:47.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:47.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:47.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:47.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:47.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:47.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:47.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:47.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:47.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:47.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:47.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:47.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:47.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:47.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:47.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:47.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:47.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:47.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:47.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:47.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:47.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:47.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:47.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:47.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:47.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:47.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:47.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:47.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:47.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:47.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:47.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:47.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:47.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:47 smithi161 ceph-mon[84406]: osdmap e179: 8 total, 8 up, 8 in 2024-06-01T02:10:48.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: pgmap v258: 161 pgs: 104 active+clean, 34 active+undersized, 22 active+undersized+degraded, 1 active+clean+remapped; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 75/726 objects degraded (10.331%); 6/726 objects misplaced (0.826%) 2024-06-01T02:10:48.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:10:48.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: osd.5 [v2:172.21.15.161:6808/3624694504,v1:172.21.15.161:6809/3624694504] boot 2024-06-01T02:10:48.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: osdmap e178: 8 total, 8 up, 8 in 2024-06-01T02:10:48.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T02:10:48.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: pgmap v258: 161 pgs: 104 active+clean, 34 active+undersized, 22 active+undersized+degraded, 1 active+clean+remapped; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 75/726 objects degraded (10.331%); 6/726 objects misplaced (0.826%) 2024-06-01T02:10:48.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:10:48.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: osd.5 [v2:172.21.15.161:6808/3624694504,v1:172.21.15.161:6809/3624694504] boot 2024-06-01T02:10:48.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: osdmap e178: 8 total, 8 up, 8 in 2024-06-01T02:10:48.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-01T02:10:48.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[89502]: osdmap e179: 8 total, 8 up, 8 in 2024-06-01T02:10:48.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:47 smithi116 ceph-mon[100606]: osdmap e179: 8 total, 8 up, 8 in 2024-06-01T02:10:48.882 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.882 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.882 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.883 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.883 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.883 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.883 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.883 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.883 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.884 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.884 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.884 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.884 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.884 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.884 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.884 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.884 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.885 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.885 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.885 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.885 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.885 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.885 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.885 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.885 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.885 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.885 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.885 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.885 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.885 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.885 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.886 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.886 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.886 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.886 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.886 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.886 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.886 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.886 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.886 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.886 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.886 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.886 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.887 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.887 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.887 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.887 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.887 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:48 smithi161 ceph-mon[84406]: osdmap e180: 8 total, 8 up, 8 in 2024-06-01T02:10:48.944 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.944 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.944 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.945 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.945 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.945 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.945 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.945 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.945 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.945 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.945 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.945 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.945 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.945 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.945 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.945 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.945 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.946 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.946 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.946 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.946 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.946 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.946 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.946 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.946 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.946 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.946 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.946 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.946 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.947 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.947 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.947 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.947 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.947 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.947 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.947 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.947 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.947 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.947 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.947 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.948 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.948 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.948 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.948 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.948 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.948 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:48.948 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.948 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.948 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.948 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.948 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.948 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.948 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.948 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.949 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.949 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.949 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.949 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.949 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.949 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.949 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.949 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.949 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.949 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.950 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.950 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.950 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.950 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.950 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.950 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.950 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.950 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.950 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.950 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.950 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.950 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.950 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[100606]: osdmap e180: 8 total, 8 up, 8 in 2024-06-01T02:10:48.951 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.951 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.951 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.951 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.951 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.951 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.951 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.951 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.951 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.951 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.951 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.951 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.952 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.952 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:48.952 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:48.952 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:48.952 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:48.952 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:48.952 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:48 smithi116 ceph-mon[89502]: osdmap e180: 8 total, 8 up, 8 in 2024-06-01T02:10:49.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:49.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:49.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: pgmap v261: 161 pgs: 1 remapped+peering, 30 peering, 109 active+clean, 12 active+undersized, 9 active+undersized+degraded; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 31/726 objects degraded (4.270%) 2024-06-01T02:10:49.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:49.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:49.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:49.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:49.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:49.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:49.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:49.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:49.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:49.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:49.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:49.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:49.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:49.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:49.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:49.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:49.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:49.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:49.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:49.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:49.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:49.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:49.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:49.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:49.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:49.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:49.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:49.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:49.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:49.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:49.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:49.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:49.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:49.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:49.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:49.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:49.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:49.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:49.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:49.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:49.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:49 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:50.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:50.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:50.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: pgmap v261: 161 pgs: 1 remapped+peering, 30 peering, 109 active+clean, 12 active+undersized, 9 active+undersized+degraded; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 31/726 objects degraded (4.270%) 2024-06-01T02:10:50.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:50.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:50.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:50.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:50.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:50.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:50.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:50.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:50.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:50.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:50.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:50.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:50.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:50.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:50.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:50.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:50.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:50.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:50.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:50.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:50.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:50.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:50.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:50.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:50.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:50.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:50.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:50.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:50.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: pgmap v261: 161 pgs: 1 remapped+peering, 30 peering, 109 active+clean, 12 active+undersized, 9 active+undersized+degraded; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 31/726 objects degraded (4.270%) 2024-06-01T02:10:50.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:50.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:50.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:50.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:50.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:50.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:50.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:50.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:50.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:50.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:50.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:50.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:50.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:50.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:50.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:50.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:50.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:50.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:50.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:50.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:50.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:50.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:50.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:50.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:50.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:50.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:50.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:50.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:50.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:50.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:50.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:50.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:50.048 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:50.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:50.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:50.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:50.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:50.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:50.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:50.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:50.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:50.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:50.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:50.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:50.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:50.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:50.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:50.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:50.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:50.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:50.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:50.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:50.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:50.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:50.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:50.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:49 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:50.944 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:10:50.788+0000 7fcedaabb700 -1 mgr.server reply reply (16) Device or resource busy unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-06-01T02:10:50.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:50.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:50.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:50.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:50.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:50.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:50.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:50.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:50.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:50.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:50.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:50.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:50.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:50.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:50.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:50.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:50.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:50.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:50.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:50.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:50.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:50.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:50.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:50.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:50.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:50.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:50.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:50.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:50.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:50.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:50.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:50.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:50.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:50.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:50.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:50.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:50.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:50.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:50.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:50.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:50.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:50.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:50.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:50.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:50.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:50.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:50.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:50.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:50.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:50.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:50.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:50.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:50.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:51.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:51.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:51.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:51.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:51.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:51.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:51.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:51.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:51.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:51.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:51.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:51.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:51.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:51.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:51.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:51.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:51.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:51.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:51.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:51.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:51.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:51.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:51.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:51.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:51.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:51.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:51.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:51.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:51.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:51.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:51.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:51.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:51.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:51.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:51.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:51.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:51.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:51.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:51.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:51.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:51.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:51.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:51.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:51.046 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:51.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:51.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:51.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:51.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:51.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:51.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:51.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:51.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:51.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:51.047 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:51.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:51.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:51.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:51.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:51.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:51.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:51.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:51.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:51.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:51.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:51.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:51.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:51.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:51.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:51.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:51.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:51.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:51.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:51.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:51.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:51.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:51.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:51.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:51.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:51.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:51.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:51.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:51.050 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:51.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:51.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:51.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:51.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:51.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:51.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:51.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:51.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:51.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:51.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:51.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:51.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:51.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:51.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:51.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:51.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:51.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:51.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:51.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:51.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:51.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:51.051 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:10:51.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:10:51.052 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:51.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:51 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:51.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:51 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:51.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:51 smithi161 ceph-mon[84406]: pgmap v263: 161 pgs: 1 remapped+peering, 30 peering, 109 active+clean, 12 active+undersized, 9 active+undersized+degraded; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 op/s; 31/726 objects degraded (4.270%) 2024-06-01T02:10:51.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:51 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:51.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:51 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:51.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:51 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:51.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:51 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:51.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:51.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:51.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:51.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:51.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:51.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:51.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:51.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:51 smithi161 ceph-mon[84406]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-06-01T02:10:51.861 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-06-01T02:10:51.861 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:51 smithi161 ceph-mon[84406]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-06-01T02:10:51.861 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:51 smithi161 ceph-mon[84406]: Health check update: Degraded data redundancy: 31/726 objects degraded (4.270%), 9 pgs degraded (PG_DEGRADED) 2024-06-01T02:10:52.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:52.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:52.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[89502]: pgmap v263: 161 pgs: 1 remapped+peering, 30 peering, 109 active+clean, 12 active+undersized, 9 active+undersized+degraded; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 op/s; 31/726 objects degraded (4.270%) 2024-06-01T02:10:52.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:52.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:52.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:52.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:52.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:52.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:52.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:52.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:52.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:52.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:52.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:52.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[89502]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-06-01T02:10:52.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-06-01T02:10:52.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[89502]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-06-01T02:10:52.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[89502]: Health check update: Degraded data redundancy: 31/726 objects degraded (4.270%), 9 pgs degraded (PG_DEGRADED) 2024-06-01T02:10:52.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:52.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:52.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[100606]: pgmap v263: 161 pgs: 1 remapped+peering, 30 peering, 109 active+clean, 12 active+undersized, 9 active+undersized+degraded; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 op/s; 31/726 objects degraded (4.270%) 2024-06-01T02:10:52.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:52.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:52.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:52.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:10:52.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:52.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:52.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:52.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:10:52.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:52.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:52.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:10:52.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[100606]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-06-01T02:10:52.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-06-01T02:10:52.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[100606]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-06-01T02:10:52.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:51 smithi116 ceph-mon[100606]: Health check update: Degraded data redundancy: 31/726 objects degraded (4.270%), 9 pgs degraded (PG_DEGRADED) 2024-06-01T02:10:52.885 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:52 smithi161 ceph-mon[84406]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 31/726 objects degraded (4.270%), 9 pgs degraded) 2024-06-01T02:10:52.885 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:52 smithi161 ceph-mon[84406]: Cluster is now healthy 2024-06-01T02:10:53.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:52 smithi116 ceph-mon[89502]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 31/726 objects degraded (4.270%), 9 pgs degraded) 2024-06-01T02:10:53.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:52 smithi116 ceph-mon[89502]: Cluster is now healthy 2024-06-01T02:10:53.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:52 smithi116 ceph-mon[100606]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 31/726 objects degraded (4.270%), 9 pgs degraded) 2024-06-01T02:10:53.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:52 smithi116 ceph-mon[100606]: Cluster is now healthy 2024-06-01T02:10:53.813 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:53 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:10:53.813 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:53 smithi161 ceph-mon[84406]: pgmap v264: 161 pgs: 18 peering, 143 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-06-01T02:10:53.813 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:53.813 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:53.813 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:54.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:53 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:10:54.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:53 smithi116 ceph-mon[89502]: pgmap v264: 161 pgs: 18 peering, 143 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-06-01T02:10:54.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:54.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:54.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:54.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:53 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:10:54.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:53 smithi116 ceph-mon[100606]: pgmap v264: 161 pgs: 18 peering, 143 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-06-01T02:10:54.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:54.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:54.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:55.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:55 smithi161 ceph-mon[84406]: pgmap v265: 161 pgs: 161 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:10:55.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:55.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:55.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:55.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:55.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:55.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:56.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:55 smithi116 ceph-mon[89502]: pgmap v265: 161 pgs: 161 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:10:56.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:56.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:56.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:56.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:56.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:56.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:56.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:55 smithi116 ceph-mon[100606]: pgmap v265: 161 pgs: 161 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:10:56.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:56.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:56.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:56.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:56.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:56.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:10:56.944 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:10:56 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:10:56] "GET /metrics HTTP/1.1" 200 38395 "" "Prometheus/2.33.4" 2024-06-01T02:10:58.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:57 smithi116 ceph-mon[89502]: pgmap v266: 161 pgs: 161 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 597 B/s rd, 0 op/s 2024-06-01T02:10:58.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:57 smithi116 ceph-mon[100606]: pgmap v266: 161 pgs: 161 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 597 B/s rd, 0 op/s 2024-06-01T02:10:58.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:57 smithi161 ceph-mon[84406]: pgmap v266: 161 pgs: 161 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 597 B/s rd, 0 op/s 2024-06-01T02:11:00.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:10:59 smithi116 ceph-mon[89502]: pgmap v267: 161 pgs: 161 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:11:00.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:10:59 smithi116 ceph-mon[100606]: pgmap v267: 161 pgs: 161 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:11:00.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:10:59 smithi161 ceph-mon[84406]: pgmap v267: 161 pgs: 161 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:11:02.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:01 smithi116 ceph-mon[89502]: pgmap v268: 161 pgs: 161 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 884 B/s rd, 0 op/s 2024-06-01T02:11:02.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:02.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:11:02.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:02.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:01 smithi116 ceph-mon[100606]: pgmap v268: 161 pgs: 161 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 884 B/s rd, 0 op/s 2024-06-01T02:11:02.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:01 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:02.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:01 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:11:02.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:01 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:02.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:01 smithi161 ceph-mon[84406]: pgmap v268: 161 pgs: 161 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 884 B/s rd, 0 op/s 2024-06-01T02:11:02.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:01 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:02.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:01 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:11:02.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:01 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:03.994 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:03 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:11:03.994 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:03 smithi116 ceph-mon[100606]: pgmap v269: 161 pgs: 161 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:11:03.994 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:03 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:11:03.994 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:03 smithi116 ceph-mon[89502]: pgmap v269: 161 pgs: 161 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:11:04.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:03 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:11:04.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:03 smithi161 ceph-mon[84406]: pgmap v269: 161 pgs: 161 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:11:06.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:05 smithi161 ceph-mon[84406]: pgmap v270: 161 pgs: 161 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:11:06.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:05 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-06-01T02:11:06.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:05 smithi116 ceph-mon[89502]: pgmap v270: 161 pgs: 161 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:11:06.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:05 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-06-01T02:11:06.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:05 smithi116 ceph-mon[100606]: pgmap v270: 161 pgs: 161 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:11:06.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:05 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-06-01T02:11:07.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:06 smithi161 ceph-mon[84406]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-06-01T02:11:07.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:06 smithi161 ceph-mon[84406]: Upgrade: osd.6 is safe to restart 2024-06-01T02:11:07.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:11:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:11:06] "GET /metrics HTTP/1.1" 200 38541 "" "Prometheus/2.33.4" 2024-06-01T02:11:07.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:06 smithi116 ceph-mon[89502]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-06-01T02:11:07.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:06 smithi116 ceph-mon[89502]: Upgrade: osd.6 is safe to restart 2024-06-01T02:11:07.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:06 smithi116 ceph-mon[100606]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-06-01T02:11:07.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:06 smithi116 ceph-mon[100606]: Upgrade: osd.6 is safe to restart 2024-06-01T02:11:08.187 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:07 smithi161 ceph-mon[84406]: pgmap v271: 161 pgs: 161 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:11:08.187 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:08.187 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-01T02:11:08.187 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:07 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:08.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:07 smithi116 ceph-mon[89502]: pgmap v271: 161 pgs: 161 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:11:08.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:08.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-01T02:11:08.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:07 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:08.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:07 smithi116 ceph-mon[100606]: pgmap v271: 161 pgs: 161 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:11:08.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:08.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-01T02:11:08.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:07 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:09.047 INFO:teuthology.orchestra.run.smithi116.stdout:true 2024-06-01T02:11:09.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:08 smithi161 ceph-mon[84406]: Upgrade: Updating osd.6 2024-06-01T02:11:09.111 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:08 smithi161 ceph-mon[84406]: Deploying daemon osd.6 on smithi161 2024-06-01T02:11:09.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:08 smithi116 ceph-mon[89502]: Upgrade: Updating osd.6 2024-06-01T02:11:09.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:08 smithi116 ceph-mon[89502]: Deploying daemon osd.6 on smithi161 2024-06-01T02:11:09.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:08 smithi116 ceph-mon[100606]: Upgrade: Updating osd.6 2024-06-01T02:11:09.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:08 smithi116 ceph-mon[100606]: Deploying daemon osd.6 on smithi161 2024-06-01T02:11:09.694 INFO:teuthology.orchestra.run.smithi116.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-01T02:11:09.695 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi116 smithi116 running 16s ago 19m - - 2024-06-01T02:11:09.695 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi161 smithi161 running 14s ago 19m - - 2024-06-01T02:11:09.695 INFO:teuthology.orchestra.run.smithi116.stdout:alertmanager.a smithi116 *:9093,9094 running (9m) 16s ago 14m 22.0M - 0.23.0 ba2b418f427c e0c297162ded 2024-06-01T02:11:09.695 INFO:teuthology.orchestra.run.smithi116.stdout:grafana.a smithi161 *:3000 running (8m) 14s ago 14m 52.0M - 8.3.5 dad864ee21e9 00b0a78ee88e 2024-06-01T02:11:09.695 INFO:teuthology.orchestra.run.smithi116.stdout:iscsi.foo.smithi116.gjuxzb smithi116 running (7m) 16s ago 12m 62.9M - 3.5 e1d6a67b021e 0e191fe3e3bb 2024-06-01T02:11:09.695 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.x smithi161 *:8443,9283,8765 running (8m) 14s ago 18m 532M - 19.0.0-2946-g3d89accf b045ac545f82 2d7fbb85e5f7 2024-06-01T02:11:09.695 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.y smithi116 *:8443,9283,8765 running (7m) 16s ago 20m 455M - 19.0.0-2946-g3d89accf b045ac545f82 bc9627c27a57 2024-06-01T02:11:09.695 INFO:teuthology.orchestra.run.smithi116.stdout:mon.a smithi116 running (6m) 16s ago 20m 70.8M 2048M 19.0.0-2946-g3d89accf b045ac545f82 b5e9b6f873ff 2024-06-01T02:11:09.695 INFO:teuthology.orchestra.run.smithi116.stdout:mon.b smithi161 running (5m) 14s ago 19m 48.8M 2048M 19.0.0-2946-g3d89accf b045ac545f82 aaa08c03cfc8 2024-06-01T02:11:09.695 INFO:teuthology.orchestra.run.smithi116.stdout:mon.c smithi116 running (5m) 16s ago 19m 54.0M 2048M 19.0.0-2946-g3d89accf b045ac545f82 bd0fb0e603fa 2024-06-01T02:11:09.695 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.a smithi116 *:9100 running (9m) 16s ago 14m 18.3M - 1.3.1 1dbe0e931976 e347748d17bb 2024-06-01T02:11:09.696 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.b smithi161 *:9100 running (8m) 14s ago 14m 17.6M - 1.3.1 1dbe0e931976 6e66182f5a1a 2024-06-01T02:11:09.696 INFO:teuthology.orchestra.run.smithi116.stdout:osd.0 smithi116 running (4m) 16s ago 18m 52.7M 4096M 19.0.0-2946-g3d89accf b045ac545f82 b4313d121b9d 2024-06-01T02:11:09.696 INFO:teuthology.orchestra.run.smithi116.stdout:osd.1 smithi116 running (3m) 16s ago 18m 54.4M 4096M 19.0.0-2946-g3d89accf b045ac545f82 dccdbeac5aae 2024-06-01T02:11:09.696 INFO:teuthology.orchestra.run.smithi116.stdout:osd.2 smithi116 running (2m) 16s ago 17m 48.9M 4096M 19.0.0-2946-g3d89accf b045ac545f82 1ad95efda404 2024-06-01T02:11:09.696 INFO:teuthology.orchestra.run.smithi116.stdout:osd.3 smithi116 running (2m) 16s ago 17m 52.1M 4096M 19.0.0-2946-g3d89accf b045ac545f82 bd00ad2c2556 2024-06-01T02:11:09.696 INFO:teuthology.orchestra.run.smithi116.stdout:osd.4 smithi161 running (96s) 14s ago 17m 50.2M 4096M 19.0.0-2946-g3d89accf b045ac545f82 0084b9edf7a0 2024-06-01T02:11:09.696 INFO:teuthology.orchestra.run.smithi116.stdout:osd.5 smithi161 running (28s) 14s ago 16m 47.1M 4096M 19.0.0-2946-g3d89accf b045ac545f82 b3849cdeb707 2024-06-01T02:11:09.696 INFO:teuthology.orchestra.run.smithi116.stdout:osd.6 smithi161 running (16m) 14s ago 16m 62.0M 4096M 17.2.0 e1d6a67b021e 50abeb0551e7 2024-06-01T02:11:09.696 INFO:teuthology.orchestra.run.smithi116.stdout:osd.7 smithi161 running (15m) 14s ago 15m 66.1M 4096M 17.2.0 e1d6a67b021e 6004e6983735 2024-06-01T02:11:09.696 INFO:teuthology.orchestra.run.smithi116.stdout:prometheus.a smithi161 *:9095 running (7m) 14s ago 15m 79.0M - 2.33.4 514e6a882f6e e977f6ca1d73 2024-06-01T02:11:09.696 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi116.vzjrxj smithi116 *:8000 running (13m) 16s ago 13m 77.7M - 17.2.0 e1d6a67b021e df4fe3c1dbc3 2024-06-01T02:11:09.696 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi161.ashqxz smithi161 *:8000 running (13m) 14s ago 13m 78.3M - 17.2.0 e1d6a67b021e 3eecc1f3e2d3 2024-06-01T02:11:09.696 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi116.hokbmf smithi116 *:80 running (13m) 16s ago 13m 78.0M - 17.2.0 e1d6a67b021e 10950ed75dbd 2024-06-01T02:11:09.697 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi161.axjlzw smithi161 *:80 running (12m) 14s ago 12m 77.5M - 17.2.0 e1d6a67b021e 0b5d5ce47772 2024-06-01T02:11:10.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:09 smithi116 ceph-mon[89502]: pgmap v272: 161 pgs: 161 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:11:10.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:09 smithi116 ceph-mon[100606]: pgmap v272: 161 pgs: 161 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:11:10.088 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:11:10.088 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": { 2024-06-01T02:11:10.088 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 3 2024-06-01T02:11:10.088 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:11:10.088 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": { 2024-06-01T02:11:10.088 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-01T02:11:10.088 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:11:10.088 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": { 2024-06-01T02:11:10.088 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 2, 2024-06-01T02:11:10.088 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 6 2024-06-01T02:11:10.088 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:11:10.088 INFO:teuthology.orchestra.run.smithi116.stdout: "rgw": { 2024-06-01T02:11:10.088 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-06-01T02:11:10.089 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:11:10.089 INFO:teuthology.orchestra.run.smithi116.stdout: "overall": { 2024-06-01T02:11:10.089 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 6, 2024-06-01T02:11:10.089 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 11 2024-06-01T02:11:10.089 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T02:11:10.089 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:11:10.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:09 smithi161 ceph-mon[84406]: pgmap v272: 161 pgs: 161 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:11:10.417 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:11:10.417 INFO:teuthology.orchestra.run.smithi116.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-01T02:11:10.417 INFO:teuthology.orchestra.run.smithi116.stdout: "in_progress": true, 2024-06-01T02:11:10.417 INFO:teuthology.orchestra.run.smithi116.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-01T02:11:10.417 INFO:teuthology.orchestra.run.smithi116.stdout: "services_complete": [ 2024-06-01T02:11:10.417 INFO:teuthology.orchestra.run.smithi116.stdout: "mon", 2024-06-01T02:11:10.417 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr" 2024-06-01T02:11:10.417 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-06-01T02:11:10.417 INFO:teuthology.orchestra.run.smithi116.stdout: "progress": "11/23 daemons upgraded", 2024-06-01T02:11:10.417 INFO:teuthology.orchestra.run.smithi116.stdout: "message": "Currently upgrading osd daemons", 2024-06-01T02:11:10.417 INFO:teuthology.orchestra.run.smithi116.stdout: "is_paused": false 2024-06-01T02:11:10.417 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:11:10.805 INFO:teuthology.orchestra.run.smithi116.stdout:HEALTH_OK 2024-06-01T02:11:11.080 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:10 smithi161 systemd[1]: Stopping Ceph osd.6 for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:11:11.081 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:10 smithi161 ceph-mon[84406]: from='client.44547 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:11:11.081 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:10 smithi161 ceph-mon[84406]: from='client.44553 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:11:11.081 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:10 smithi161 ceph-mon[84406]: from='client.44559 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:11:11.081 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:10 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/3275950670' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:11.081 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:10 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/3584790290' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:11:11.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:10 smithi116 ceph-mon[89502]: from='client.44547 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:11:11.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:10 smithi116 ceph-mon[89502]: from='client.44553 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:11:11.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:10 smithi116 ceph-mon[89502]: from='client.44559 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:11:11.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:10 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/3275950670' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:11.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:10 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/3584790290' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:11:11.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:10 smithi116 ceph-mon[100606]: from='client.44547 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:11:11.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:10 smithi116 ceph-mon[100606]: from='client.44553 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:11:11.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:10 smithi116 ceph-mon[100606]: from='client.44559 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:11:11.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:10 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/3275950670' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:11.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:10 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/3584790290' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:11:11.444 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:11 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[36538]: 2024-06-01T02:11:11.108+0000 7f63d8805700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-01T02:11:11.444 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:11 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[36538]: 2024-06-01T02:11:11.108+0000 7f63d8805700 -1 osd.6 180 *** Got signal Terminated *** 2024-06-01T02:11:11.444 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:11 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[36538]: 2024-06-01T02:11:11.108+0000 7f63d8805700 -1 osd.6 180 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-01T02:11:12.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:11 smithi161 ceph-mon[84406]: pgmap v273: 161 pgs: 161 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:11:12.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:11 smithi161 ceph-mon[84406]: from='client.44571 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:11:12.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:11 smithi161 ceph-mon[84406]: osd.6 marked itself down and dead 2024-06-01T02:11:12.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:11 smithi116 ceph-mon[89502]: pgmap v273: 161 pgs: 161 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:11:12.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:11 smithi116 ceph-mon[89502]: from='client.44571 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:11:12.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:11 smithi116 ceph-mon[89502]: osd.6 marked itself down and dead 2024-06-01T02:11:12.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:11 smithi116 ceph-mon[100606]: pgmap v273: 161 pgs: 161 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:11:12.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:11 smithi116 ceph-mon[100606]: from='client.44571 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:11:12.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:11 smithi116 ceph-mon[100606]: osd.6 marked itself down and dead 2024-06-01T02:11:12.694 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:12 smithi161 podman[118985]: 2024-06-01 02:11:12.57673782 +0000 UTC m=+1.573261535 container died 50abeb0551e769bac67a383193e78b9eb5ac216ea8d1ad0468f9082e6b65d989 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, name=centos-stream, GIT_BRANCH=HEAD, vendor=Red Hat, Inc., io.openshift.tags=base centos centos-stream, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , release=754, vcs-type=git, architecture=x86_64, io.buildah.version=1.19.8, io.openshift.expose-services=, CEPH_POINT_RELEASE=-17.2.0, RELEASE=HEAD, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., GIT_REPO=https://github.com/ceph/ceph-container.git, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, com.redhat.component=centos-stream-container, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, io.k8s.display-name=CentOS Stream 8, version=8, build-date=2022-05-03T08:36:31.336870, distribution-scope=public, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream) 2024-06-01T02:11:13.144 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:12 smithi161 ceph-mon[84406]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:11:13.144 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:12 smithi161 ceph-mon[84406]: osdmap e181: 8 total, 7 up, 8 in 2024-06-01T02:11:13.145 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:13 smithi161 podman[118985]: 2024-06-01 02:11:13.132457246 +0000 UTC m=+2.128980945 container cleanup 50abeb0551e769bac67a383193e78b9eb5ac216ea8d1ad0468f9082e6b65d989 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6, vendor=Red Hat, Inc., io.openshift.expose-services=, distribution-scope=public, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, build-date=2022-05-03T08:36:31.336870, maintainer=Guillaume Abrioux , name=centos-stream, RELEASE=HEAD, architecture=x86_64, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.k8s.display-name=CentOS Stream 8, release=754, version=8, CEPH_POINT_RELEASE=-17.2.0, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, io.openshift.tags=base centos centos-stream, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, ceph=True, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, GIT_BRANCH=HEAD, com.redhat.component=centos-stream-container, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, vcs-type=git, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, io.buildah.version=1.19.8) 2024-06-01T02:11:13.145 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:13 smithi161 bash[118985]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6 2024-06-01T02:11:13.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:12 smithi116 ceph-mon[100606]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:11:13.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:12 smithi116 ceph-mon[100606]: osdmap e181: 8 total, 7 up, 8 in 2024-06-01T02:11:13.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:12 smithi116 ceph-mon[89502]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:11:13.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:12 smithi116 ceph-mon[89502]: osdmap e181: 8 total, 7 up, 8 in 2024-06-01T02:11:13.694 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:13 smithi161 podman[119194]: 2024-06-01 02:11:13.359014237 +0000 UTC m=+1.107732700 container remove 50abeb0551e769bac67a383193e78b9eb5ac216ea8d1ad0468f9082e6b65d989 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, release=754, architecture=x86_64, io.k8s.display-name=CentOS Stream 8, io.openshift.expose-services=, CEPH_POINT_RELEASE=-17.2.0, io.buildah.version=1.19.8, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, build-date=2022-05-03T08:36:31.336870, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, vcs-type=git, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, RELEASE=HEAD, maintainer=Guillaume Abrioux , version=8, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, GIT_BRANCH=HEAD, vendor=Red Hat, Inc., GIT_CLEAN=True, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., GIT_REPO=https://github.com/ceph/ceph-container.git, com.redhat.component=centos-stream-container, distribution-scope=public, io.openshift.tags=base centos centos-stream, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, name=centos-stream, ceph=True) 2024-06-01T02:11:14.067 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:13 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:11:14.067 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:13 smithi161 ceph-mon[84406]: pgmap v275: 161 pgs: 14 stale+active+clean, 147 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:11:14.067 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:13 smithi161 ceph-mon[84406]: osdmap e182: 8 total, 7 up, 8 in 2024-06-01T02:11:14.067 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:14.067 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:14.068 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:13 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:14.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:13 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:11:14.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:13 smithi116 ceph-mon[89502]: pgmap v275: 161 pgs: 14 stale+active+clean, 147 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:11:14.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:13 smithi116 ceph-mon[89502]: osdmap e182: 8 total, 7 up, 8 in 2024-06-01T02:11:14.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:14.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:14.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:13 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:14.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:13 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:11:14.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:13 smithi116 ceph-mon[100606]: pgmap v275: 161 pgs: 14 stale+active+clean, 147 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:11:14.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:13 smithi116 ceph-mon[100606]: osdmap e182: 8 total, 7 up, 8 in 2024-06-01T02:11:14.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:14.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:14.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:13 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:15.068 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:14 smithi161 ceph-mon[84406]: Health check failed: Reduced data availability: 5 pgs inactive, 10 pgs peering (PG_AVAILABILITY) 2024-06-01T02:11:15.069 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:14 smithi161 podman[119395]: 2024-06-01 02:11:14.820316426 +0000 UTC m=+0.019518562 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:11:15.069 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:14 smithi161 podman[119395]: 2024-06-01 02:11:14.943740864 +0000 UTC m=+0.142942981 container create c053abf8be5553ddcfd6bcaa832ca2fbe9b11d50655d80b31da86af07174f84c (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, RELEASE=squid-3d89acc) 2024-06-01T02:11:15.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:14 smithi116 ceph-mon[89502]: Health check failed: Reduced data availability: 5 pgs inactive, 10 pgs peering (PG_AVAILABILITY) 2024-06-01T02:11:15.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:14 smithi116 ceph-mon[100606]: Health check failed: Reduced data availability: 5 pgs inactive, 10 pgs peering (PG_AVAILABILITY) 2024-06-01T02:11:15.444 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:15 smithi161 podman[119395]: 2024-06-01 02:11:15.171541197 +0000 UTC m=+0.370743339 container init c053abf8be5553ddcfd6bcaa832ca2fbe9b11d50655d80b31da86af07174f84c (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-06-01T02:11:15.444 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:15 smithi161 podman[119395]: 2024-06-01 02:11:15.175061296 +0000 UTC m=+0.374263422 container start c053abf8be5553ddcfd6bcaa832ca2fbe9b11d50655d80b31da86af07174f84c (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-06-01T02:11:15.444 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:15 smithi161 podman[119395]: 2024-06-01 02:11:15.263258365 +0000 UTC m=+0.462460533 container attach c053abf8be5553ddcfd6bcaa832ca2fbe9b11d50655d80b31da86af07174f84c (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-06-01T02:11:15.779 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:15 smithi161 podman[119395]: 2024-06-01 02:11:15.503000836 +0000 UTC m=+0.702202984 container died c053abf8be5553ddcfd6bcaa832ca2fbe9b11d50655d80b31da86af07174f84c (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=) 2024-06-01T02:11:16.033 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:15 smithi161 ceph-mon[84406]: pgmap v277: 161 pgs: 36 peering, 3 stale+active+clean, 122 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:11:16.033 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:16.033 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:16.033 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:16.033 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:16.033 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:11:16.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:15 smithi116 ceph-mon[89502]: pgmap v277: 161 pgs: 36 peering, 3 stale+active+clean, 122 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:11:16.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:16.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:16.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:16.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:16.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:11:16.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:15 smithi116 ceph-mon[100606]: pgmap v277: 161 pgs: 36 peering, 3 stale+active+clean, 122 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:11:16.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:16.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:16.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:16.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:16.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:11:17.059 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:11:16 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:11:16] "GET /metrics HTTP/1.1" 200 38533 "" "Prometheus/2.33.4" 2024-06-01T02:11:17.060 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:16 smithi161 ceph-mon[84406]: pgmap v278: 161 pgs: 4 active+undersized+degraded, 6 active+undersized, 36 peering, 1 stale+active+clean, 114 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 13/726 objects degraded (1.791%) 2024-06-01T02:11:17.060 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:16 smithi161 ceph-mon[84406]: Health check failed: Degraded data redundancy: 13/726 objects degraded (1.791%), 4 pgs degraded (PG_DEGRADED) 2024-06-01T02:11:17.060 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:16 smithi161 podman[119498]: 2024-06-01 02:11:16.829949318 +0000 UTC m=+1.321350205 container remove c053abf8be5553ddcfd6bcaa832ca2fbe9b11d50655d80b31da86af07174f84c (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-01T02:11:17.060 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:16 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.6.service: Deactivated successfully. 2024-06-01T02:11:17.060 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:16 smithi161 systemd[1]: Stopped Ceph osd.6 for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:11:17.060 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:16 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.6.service: Consumed 11.418s CPU time. 2024-06-01T02:11:17.060 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:17 smithi161 systemd[1]: Starting Ceph osd.6 for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:11:17.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:16 smithi116 ceph-mon[89502]: pgmap v278: 161 pgs: 4 active+undersized+degraded, 6 active+undersized, 36 peering, 1 stale+active+clean, 114 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 13/726 objects degraded (1.791%) 2024-06-01T02:11:17.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:16 smithi116 ceph-mon[89502]: Health check failed: Degraded data redundancy: 13/726 objects degraded (1.791%), 4 pgs degraded (PG_DEGRADED) 2024-06-01T02:11:17.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:16 smithi116 ceph-mon[100606]: pgmap v278: 161 pgs: 4 active+undersized+degraded, 6 active+undersized, 36 peering, 1 stale+active+clean, 114 active+clean; 585 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 13/726 objects degraded (1.791%) 2024-06-01T02:11:17.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:16 smithi116 ceph-mon[100606]: Health check failed: Degraded data redundancy: 13/726 objects degraded (1.791%), 4 pgs degraded (PG_DEGRADED) 2024-06-01T02:11:18.090 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:17 smithi161 podman[119747]: 2024-06-01 02:11:17.654353251 +0000 UTC m=+0.020665177 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:11:18.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:18 smithi161 ceph-mon[84406]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs inactive, 10 pgs peering) 2024-06-01T02:11:18.792 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:18 smithi116 ceph-mon[89502]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs inactive, 10 pgs peering) 2024-06-01T02:11:18.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:18 smithi116 ceph-mon[100606]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs inactive, 10 pgs peering) 2024-06-01T02:11:18.878 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:18 smithi161 podman[119747]: 2024-06-01 02:11:18.61382156 +0000 UTC m=+0.980133478 container create 0464c242ca8ed14a7bcc2e3397f0bfeb5a58aee52bd0e7abe9c494c468e331d4 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6-activate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-06-01T02:11:19.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:19 smithi161 ceph-mon[84406]: pgmap v279: 161 pgs: 20 active+undersized+degraded, 35 active+undersized, 106 active+clean; 585 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 87/726 objects degraded (11.983%) 2024-06-01T02:11:19.694 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:19 smithi161 podman[119747]: 2024-06-01 02:11:19.478884988 +0000 UTC m=+1.845196905 container init 0464c242ca8ed14a7bcc2e3397f0bfeb5a58aee52bd0e7abe9c494c468e331d4 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6-activate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=) 2024-06-01T02:11:19.694 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:19 smithi161 podman[119747]: 2024-06-01 02:11:19.482788216 +0000 UTC m=+1.849100140 container start 0464c242ca8ed14a7bcc2e3397f0bfeb5a58aee52bd0e7abe9c494c468e331d4 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6-activate, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-06-01T02:11:19.694 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:19 smithi161 podman[119747]: 2024-06-01 02:11:19.586563852 +0000 UTC m=+1.952875772 container attach 0464c242ca8ed14a7bcc2e3397f0bfeb5a58aee52bd0e7abe9c494c468e331d4 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6-activate, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, RELEASE=squid-3d89acc) 2024-06-01T02:11:19.792 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:19 smithi116 ceph-mon[89502]: pgmap v279: 161 pgs: 20 active+undersized+degraded, 35 active+undersized, 106 active+clean; 585 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 87/726 objects degraded (11.983%) 2024-06-01T02:11:19.792 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:19 smithi116 ceph-mon[100606]: pgmap v279: 161 pgs: 20 active+undersized+degraded, 35 active+undersized, 106 active+clean; 585 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 87/726 objects degraded (11.983%) 2024-06-01T02:11:20.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:19 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6-activate[119827]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-06-01T02:11:20.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:19 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6-activate[119827]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-6 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-06-01T02:11:20.195 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:19 smithi161 bash[119747]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-06-01T02:11:20.195 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:19 smithi161 bash[119747]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-6 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-06-01T02:11:20.195 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:19 smithi161 bash[119747]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-06-01T02:11:20.195 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:19 smithi161 bash[119747]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-06-01T02:11:20.195 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:19 smithi161 bash[119747]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-6/block 2024-06-01T02:11:20.195 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:19 smithi161 bash[119747]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-06-01T02:11:20.195 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:19 smithi161 bash[119747]: --> ceph-volume raw activate successful for osd ID: 6 2024-06-01T02:11:20.195 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:19 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6-activate[119827]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-06-01T02:11:20.195 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:19 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6-activate[119827]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-06-01T02:11:20.195 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:19 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6-activate[119827]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-6/block 2024-06-01T02:11:20.195 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:19 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6-activate[119827]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-06-01T02:11:20.195 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:19 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6-activate[119827]: --> ceph-volume raw activate successful for osd ID: 6 2024-06-01T02:11:20.195 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:19 smithi161 podman[119747]: 2024-06-01 02:11:19.960032927 +0000 UTC m=+2.326344846 container died 0464c242ca8ed14a7bcc2e3397f0bfeb5a58aee52bd0e7abe9c494c468e331d4 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-06-01T02:11:20.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:20.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:20.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:20 smithi161 ceph-mon[84406]: pgmap v280: 161 pgs: 20 active+undersized+degraded, 35 active+undersized, 106 active+clean; 585 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 87/726 objects degraded (11.983%) 2024-06-01T02:11:20.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:20.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:20.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:20.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:20 smithi116 ceph-mon[89502]: pgmap v280: 161 pgs: 20 active+undersized+degraded, 35 active+undersized, 106 active+clean; 585 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 87/726 objects degraded (11.983%) 2024-06-01T02:11:20.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:20.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:20.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:20.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:20 smithi116 ceph-mon[100606]: pgmap v280: 161 pgs: 20 active+undersized+degraded, 35 active+undersized, 106 active+clean; 585 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 87/726 objects degraded (11.983%) 2024-06-01T02:11:20.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:20.919 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:20 smithi161 podman[120054]: 2024-06-01 02:11:20.615862279 +0000 UTC m=+0.649730421 container remove 0464c242ca8ed14a7bcc2e3397f0bfeb5a58aee52bd0e7abe9c494c468e331d4 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6-activate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-06-01T02:11:20.919 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:20 smithi161 podman[120114]: 2024-06-01 02:11:20.825156922 +0000 UTC m=+0.112148594 container create 6b87d9299c8d26ff1466530771c14803eba2d44735d55e1437793c1380c1e2e6 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-06-01T02:11:20.919 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:20 smithi161 podman[120114]: 2024-06-01 02:11:20.730699197 +0000 UTC m=+0.017690879 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:11:21.298 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:21 smithi161 podman[120114]: 2024-06-01 02:11:21.019181832 +0000 UTC m=+0.306173514 container init 6b87d9299c8d26ff1466530771c14803eba2d44735d55e1437793c1380c1e2e6 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD) 2024-06-01T02:11:21.299 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:21 smithi161 podman[120114]: 2024-06-01 02:11:21.023167595 +0000 UTC m=+0.310159265 container start 6b87d9299c8d26ff1466530771c14803eba2d44735d55e1437793c1380c1e2e6 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6, org.label-schema.build-date=20240528, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-06-01T02:11:21.299 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:21 smithi161 ceph-osd[120133]: -- 172.21.15.161:0/3051556702 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55616d189680 con 0x55616d153000 2024-06-01T02:11:21.299 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:21 smithi161 bash[120114]: 6b87d9299c8d26ff1466530771c14803eba2d44735d55e1437793c1380c1e2e6 2024-06-01T02:11:21.299 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:21 smithi161 systemd[1]: Started Ceph osd.6 for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:11:21.556 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:21 smithi161 ceph-mon[84406]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-06-01T02:11:21.556 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:21.556 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:21.556 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:21.556 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:21.556 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:21.556 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:21.556 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:21.556 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:21.556 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:21.556 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:21 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:21.790 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:21 smithi116 ceph-mon[100606]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-06-01T02:11:21.790 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:21.790 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:21.790 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:21.790 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:21.790 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:21.791 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:21.791 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:21.791 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:21.791 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:21.791 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:21 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:21.791 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:21 smithi116 ceph-mon[89502]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-06-01T02:11:21.791 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:21.791 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:21.791 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:21.791 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:21.792 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:21.792 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:21.792 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:21.792 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:21.792 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:21.792 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:21 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:21.913 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:21 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:11:21.560+0000 7f73428637c0 -1 Falling back to public interface 2024-06-01T02:11:22.650 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: pgmap v281: 161 pgs: 20 active+undersized+degraded, 35 active+undersized, 106 active+clean; 585 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 87/726 objects degraded (11.983%) 2024-06-01T02:11:22.650 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:22.650 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:22.650 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:22.650 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:22.650 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:22.650 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:22.650 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:22.650 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:22.650 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:22.650 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:22.651 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:22.651 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:22.651 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:22.651 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:22.651 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:22.651 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:22.651 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:22.651 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:22.651 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:22.651 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:22.651 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:22.651 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:22.651 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:22.652 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:22.652 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:22.652 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:22.652 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:22.652 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:22.652 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:22.652 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:22.652 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:22.652 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-06-01T02:11:22.652 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:22.652 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:22.652 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:22.652 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:22.653 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:22.653 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:22.653 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:22.653 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:22.653 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:22.653 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:22.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: pgmap v281: 161 pgs: 20 active+undersized+degraded, 35 active+undersized, 106 active+clean; 585 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 87/726 objects degraded (11.983%) 2024-06-01T02:11:22.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:22.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:22.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:22.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:22.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:22.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:22.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:22.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:22.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:22.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:22.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:22.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:22.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:22.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:22.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:22.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:22.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:22.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:22.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:22.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:22.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:22.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:22.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:22.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:22.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:22.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:22.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:22.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: pgmap v281: 161 pgs: 20 active+undersized+degraded, 35 active+undersized, 106 active+clean; 585 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 87/726 objects degraded (11.983%) 2024-06-01T02:11:22.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:22.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:22.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:22.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:22.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:22.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:22.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:22.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:22.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:22.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:22.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:22.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:22.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:22.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:22.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:22.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:22.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:22.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:22.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:22.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:22.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:22.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:22.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:22.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:22.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:22.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:22.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:22.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:22.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:22.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:22.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:22.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-06-01T02:11:22.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:22.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:22.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:22.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:22.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:22.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:22.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:22.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:22.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:22.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:22.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:22.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:22.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:22.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:22.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-06-01T02:11:22.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:22.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:22.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:22.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:22.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:22.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:22.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:22.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:22.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:22.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:23.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:23.723 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:11:23.723 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:23.723 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:23.723 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:23.723 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:23.723 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:23.723 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:23.723 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:23.723 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:23.723 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:23.723 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:23.723 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:23.723 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:23.724 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:23.724 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:23.724 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:23.724 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:23.724 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:23.724 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:23.724 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:23.724 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:23.724 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:23.724 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:23.724 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:23.724 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:23.724 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:23.724 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:23.725 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:23.725 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:23.725 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:23.725 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:23.725 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:23.725 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:23.725 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:23.725 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:23.725 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:23.725 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:23.725 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:23.725 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:23.725 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:23.726 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:23.726 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:23.726 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:23.726 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:23.726 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:23.726 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:23.726 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:23.726 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:23.726 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:23.726 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:23.726 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:23.726 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:23.726 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:23.727 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:23.727 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:23.727 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:23.727 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:23.727 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:23.782 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:23.782 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:11:23.783 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:23.783 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:23.783 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:23.783 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:23.783 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:23.783 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:23.783 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:23.783 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:23.783 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:23.783 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:23.783 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:23.783 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:23.783 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:23.784 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:23.784 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:23.784 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:23.784 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:23.784 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:23.784 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:23.784 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:23.784 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:23.784 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:23.784 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:23.784 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:23.784 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:23.784 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:23.784 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:23.785 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:23.785 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:11:23.785 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:23.785 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:23.785 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:23.785 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:23.785 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:23.785 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:23.785 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:23.785 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:23.786 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:23.786 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:23.786 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:23.786 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:23.786 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:23.786 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:23.786 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:23.786 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:23.786 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:23.786 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:23.786 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:23.786 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:23.787 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:23.787 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:23.787 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:23.787 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:23.787 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:23.787 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:23.787 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:23.788 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:23.788 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:23.788 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:23.788 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:23.788 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:23.788 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:23.788 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:23.788 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:23.788 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:23.788 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:23.788 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:23.788 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:23.788 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:23.789 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:23.789 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:23.789 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:23.789 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:23.789 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:23.789 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:23.789 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:23.789 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:23.789 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:23.789 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:23.789 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:23.789 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:23.789 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:23.789 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:23.790 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:23.790 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:23.790 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:23.790 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:23.790 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:23.790 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:23.790 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:23.790 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:23.790 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:23.791 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:23.791 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:23.791 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:23.791 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:23.791 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:23.791 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:23.791 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:23.791 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:23.791 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:23.791 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:23.791 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:23.791 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:23.791 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:23.791 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:23.791 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:23.792 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:23.792 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:23.792 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:23.792 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:23.792 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:23.792 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:23.792 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:23.792 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:23.792 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:24.295 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:11:24.280+0000 7f73428637c0 -1 osd.6 0 read_superblock omap replica is missing. 2024-06-01T02:11:24.657 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:11:24.318+0000 7f73428637c0 -1 osd.6 180 log_to_monitors true 2024-06-01T02:11:24.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:24.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:24.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: pgmap v282: 161 pgs: 20 active+undersized+degraded, 35 active+undersized, 106 active+clean; 585 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 993 B/s rd, 0 op/s; 87/726 objects degraded (11.983%) 2024-06-01T02:11:24.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:24.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:24.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:24.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:24.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:24.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:24.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:24.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:24.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:24.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:24.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:24.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:24.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:24.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:24.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:24.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:24.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:24.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:24.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:24.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:24.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:24.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:24.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:24.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:24.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:24.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:24.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:24.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:24.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:24.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:24.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:24.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:24.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:24.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:24.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:24.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:24.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:24.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:24.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:24.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:24.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:24.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:24.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:24.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='osd.6 [v2:172.21.15.161:6816/3601422775,v1:172.21.15.161:6817/3601422775]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-01T02:11:24.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:24.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:24.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:24.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:24.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:24 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:24.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:24.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:24.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: pgmap v282: 161 pgs: 20 active+undersized+degraded, 35 active+undersized, 106 active+clean; 585 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 993 B/s rd, 0 op/s; 87/726 objects degraded (11.983%) 2024-06-01T02:11:24.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:24.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:24.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:24.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:24.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:24.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:24.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:24.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:24.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:24.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:24.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:24.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:24.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:24.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:24.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:24.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:24.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:24.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:24.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:24.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:24.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:24.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:24.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:24.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:24.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:24.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:24.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:24.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:24.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: pgmap v282: 161 pgs: 20 active+undersized+degraded, 35 active+undersized, 106 active+clean; 585 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 993 B/s rd, 0 op/s; 87/726 objects degraded (11.983%) 2024-06-01T02:11:24.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:24.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:24.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:24.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:24.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:24.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:24.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:24.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:24.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:24.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:24.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:24.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:24.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:24.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:24.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:24.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:24.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:24.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:24.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:24.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:24.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:24.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:24.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:24.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:24.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:24.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:24.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:24.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:24.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:24.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:24.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:24.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:24.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:24.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:24.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:24.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:24.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:24.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:24.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:24.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:24.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:24.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:24.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:24.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='osd.6 [v2:172.21.15.161:6816/3601422775,v1:172.21.15.161:6817/3601422775]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-01T02:11:24.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:24.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:24.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:24.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:24.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:24.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:24.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:24.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:24.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:24.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:24.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:24.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:24.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:24.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:24.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:24.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:24.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:24.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:24.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:24.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:24.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:24.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:24.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='osd.6 [v2:172.21.15.161:6816/3601422775,v1:172.21.15.161:6817/3601422775]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-01T02:11:24.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:24.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:24.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:24.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:24.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:24 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:25.483 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:11:25.274+0000 7f7333cba700 -1 osd.6 180 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-01T02:11:25.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:25.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:25.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:25.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:25.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:25.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:25.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:25.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:25.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:25.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:25.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:25.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:25.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:25.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:25.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:25.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:25.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:25.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:25.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:25.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:25.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:25.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:25.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:25.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:25.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:25.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:25.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:25.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:25.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:25.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:25.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:25.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:25.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:25.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:25.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:25.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:25.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:25.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:25.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:25.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:25.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:25.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:25.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:25.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:25.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:25.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:25.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:25.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:25.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:25.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:25.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:25.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:25.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:25.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:25.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:25.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:25.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:25.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:25.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:25.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:25.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:25.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='osd.6 [v2:172.21.15.161:6816/3601422775,v1:172.21.15.161:6817/3601422775]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-01T02:11:25.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: osdmap e183: 8 total, 7 up, 8 in 2024-06-01T02:11:25.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:25.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='osd.6 [v2:172.21.15.161:6816/3601422775,v1:172.21.15.161:6817/3601422775]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:11:25.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:25.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:25.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:25.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:25.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:25.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:25.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:25.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:25.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:25.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:25.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:25.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:25.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='osd.6 [v2:172.21.15.161:6816/3601422775,v1:172.21.15.161:6817/3601422775]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-01T02:11:25.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: osdmap e183: 8 total, 7 up, 8 in 2024-06-01T02:11:25.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:25.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='osd.6 [v2:172.21.15.161:6816/3601422775,v1:172.21.15.161:6817/3601422775]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:11:25.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:25.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:25.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:25.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:25.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:25.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:25.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:25.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:25.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:25.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:25.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:25.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:25.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:25.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:25.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:25.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:25.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:25.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:25.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:25.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:25.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:25.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:25.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:25.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:25.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:25.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:25.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:25.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:25.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:25.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:25.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:25.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:25.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:25.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:25.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:25.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:25.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:25.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:25.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:25.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:25.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:25.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='osd.6 [v2:172.21.15.161:6816/3601422775,v1:172.21.15.161:6817/3601422775]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-01T02:11:25.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: osdmap e183: 8 total, 7 up, 8 in 2024-06-01T02:11:25.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:25.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='osd.6 [v2:172.21.15.161:6816/3601422775,v1:172.21.15.161:6817/3601422775]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:11:25.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:25.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:25.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:25.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:25.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:25.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:25.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:25.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:25.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:26.582 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:26.582 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: pgmap v283: 161 pgs: 20 active+undersized+degraded, 35 active+undersized, 106 active+clean; 585 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 935 B/s rd, 0 op/s; 87/726 objects degraded (11.983%) 2024-06-01T02:11:26.582 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:26.582 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:26.582 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:26.582 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:26.582 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:26.582 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:26.582 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:26.582 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:26.582 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:26.583 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:26.583 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:26.583 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:26.583 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:26.583 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:26.583 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:26.583 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:26.583 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:26.583 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:26.583 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:26.583 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:26.583 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:26.583 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:26.583 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:26.583 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:26.584 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:26.584 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:26.584 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:26.584 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:26.584 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:26.584 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:26.584 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:26.584 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:26.584 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:26.584 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:26.584 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:26.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:11:26.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: osd.6 [v2:172.21.15.161:6816/3601422775,v1:172.21.15.161:6817/3601422775] boot 2024-06-01T02:11:26.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: osdmap e184: 8 total, 8 up, 8 in 2024-06-01T02:11:26.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T02:11:26.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:26.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:26.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:26.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:26.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:26.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: Health check update: Degraded data redundancy: 87/726 objects degraded (11.983%), 20 pgs degraded (PG_DEGRADED) 2024-06-01T02:11:26.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:26.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:26.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:26.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:26.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:26.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:26.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: pgmap v283: 161 pgs: 20 active+undersized+degraded, 35 active+undersized, 106 active+clean; 585 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 935 B/s rd, 0 op/s; 87/726 objects degraded (11.983%) 2024-06-01T02:11:26.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:26.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:26.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:26.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:26.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:26.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:26.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:26.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:26.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:26.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:26.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:26.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:26.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:26.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:26.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:26.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:26.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:26.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:26.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:26.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:26.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:26.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:26.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:26.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:26.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:26.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:26.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:26.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: pgmap v283: 161 pgs: 20 active+undersized+degraded, 35 active+undersized, 106 active+clean; 585 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 935 B/s rd, 0 op/s; 87/726 objects degraded (11.983%) 2024-06-01T02:11:26.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:26.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:26.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:26.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:26.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:26.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:26.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:26.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:26.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:26.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:26.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:26.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:26.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:26.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:26.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:26.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:26.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:26.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:26.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:26.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:26.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:26.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:26.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:26.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:26.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:26.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:26.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:26.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:26.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:26.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:26.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:26.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:26.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:26.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:26.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:26.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:11:26.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: osd.6 [v2:172.21.15.161:6816/3601422775,v1:172.21.15.161:6817/3601422775] boot 2024-06-01T02:11:26.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: osdmap e184: 8 total, 8 up, 8 in 2024-06-01T02:11:26.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T02:11:26.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:26.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:26.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:26.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:26.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:26.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: Health check update: Degraded data redundancy: 87/726 objects degraded (11.983%), 20 pgs degraded (PG_DEGRADED) 2024-06-01T02:11:26.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:26.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:26.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:26.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:26.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:26.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:26.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:26.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:26.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:26.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:26.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:26.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:26.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:26.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:26.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:11:26.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: osd.6 [v2:172.21.15.161:6816/3601422775,v1:172.21.15.161:6817/3601422775] boot 2024-06-01T02:11:26.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: osdmap e184: 8 total, 8 up, 8 in 2024-06-01T02:11:26.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-01T02:11:26.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:26.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:26.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:26.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:26.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:26.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: Health check update: Degraded data redundancy: 87/726 objects degraded (11.983%), 20 pgs degraded (PG_DEGRADED) 2024-06-01T02:11:26.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:26.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:26.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:26.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:26.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:26.943 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:11:26 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:11:26] "GET /metrics HTTP/1.1" 200 38533 "" "Prometheus/2.33.4" 2024-06-01T02:11:27.747 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:27.747 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:27.747 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:27.747 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:27.748 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:27.748 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:27.748 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:27.748 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:27.748 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:27.749 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:27.749 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:27.749 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:27.749 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:27.749 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:27.749 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:27.750 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:27.750 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:27.750 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:27.750 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:27.750 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:27.750 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:27.750 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:27.751 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:27.751 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:27.751 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:27.751 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:27.751 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:27.751 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:27.751 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:27.751 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:27.752 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:27.752 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:27.752 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:27.752 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:27.752 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:27.753 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:27.753 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:27.753 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:27.753 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:27.753 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: osdmap e185: 8 total, 8 up, 8 in 2024-06-01T02:11:27.753 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:27.753 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:27.753 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:27.753 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:27.753 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:27.753 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:27.753 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:27.753 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:27.754 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:27.754 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:27.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:27.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:27.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:27.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:27.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:27.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:27.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:27.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:27.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:27.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:27.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:27.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:27.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:27.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:27.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:27.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:27.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:27.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:27.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:27.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:27.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:27.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:27.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:27.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:27.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:27.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:27.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:27.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:27.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:27.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:27.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:27.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:27.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:27.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:27.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:27.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:27.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:27.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:27.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:27.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:27.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:27.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:27.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:27.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:27.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:27.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:27.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:27.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:27.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:27.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:27.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:27.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:27.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:27.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:27.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:27.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:27.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:27.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:27.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:27.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:27.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:27.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:27.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:27.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:27.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:27.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:27.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:27.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:27.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:27.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: osdmap e185: 8 total, 8 up, 8 in 2024-06-01T02:11:27.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:27.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:27.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:27.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:27.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:27.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:27.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:27.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:27.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:27.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:27.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:27.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:27.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:27.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:27.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:27.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:27.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:27.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:27.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:27.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: osdmap e185: 8 total, 8 up, 8 in 2024-06-01T02:11:27.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:27.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:27.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:27.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:27.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:27.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:27.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:27.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:27.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:27.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:28.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:28.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: pgmap v286: 161 pgs: 8 peering, 16 active+undersized+degraded, 31 active+undersized, 106 active+clean; 585 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 664 B/s rd, 0 op/s; 70/726 objects degraded (9.642%) 2024-06-01T02:11:28.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:28.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:28.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:28.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:28.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:28.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:28.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:28.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:28.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:28.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:28.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:28.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:28.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:28.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:28.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:28.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:28.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:28.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:28.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:28.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:28.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:28.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:28.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:28.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:28.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:28.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:28.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:28.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:28.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:28.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:28.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:28.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:28.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:28.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:28.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:28.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:28.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:28.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:28.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:28.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:28.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:28.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:28.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:28.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: pgmap v286: 161 pgs: 8 peering, 16 active+undersized+degraded, 31 active+undersized, 106 active+clean; 585 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 664 B/s rd, 0 op/s; 70/726 objects degraded (9.642%) 2024-06-01T02:11:28.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:28.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:28.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:28.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:28.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:28.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:28.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:28.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:28.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:28.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:28.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:28.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:28.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:28.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:28.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:28.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:28.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:28.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:28.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:28.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:28.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:28.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:28.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:28.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:28.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:28.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:28.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:28.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:28.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: pgmap v286: 161 pgs: 8 peering, 16 active+undersized+degraded, 31 active+undersized, 106 active+clean; 585 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 664 B/s rd, 0 op/s; 70/726 objects degraded (9.642%) 2024-06-01T02:11:28.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:28.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:28.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:28.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:28.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:28.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:28.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:28.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:28.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:28.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:28.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:28.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:28.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:28.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:28.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:28.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:28.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:28.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:28.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:28.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:28.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:28.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:28.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:28.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:28.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:28.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:28.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:28.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:28.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:28.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:28.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:28.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:28.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:28.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:28.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:28.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:28.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:28.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:28.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:28.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:28.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:28.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:28.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:28.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:28.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:28.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:28.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:28.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:28.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:28.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:28.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:28.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:28.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:28.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:28.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:28.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:28.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:29.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:29.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:29.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:29.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:29.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:29.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:29.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:29.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:29.613 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:29.613 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:29.613 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:29.613 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:29.613 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:29.613 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:29.613 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:29.613 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:29.613 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:29.613 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:29.614 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:29.614 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:29.614 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:29.614 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:29.614 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:29.614 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:29.614 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:29.614 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:29.614 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:29.614 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:29.614 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:29.615 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:29.615 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:29.615 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:29.615 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:29.615 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:29.615 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:29.615 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:29.615 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:29.615 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:29.615 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:29.616 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:29.616 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:29.616 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:29.616 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:29.616 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:29.616 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:29.616 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:29.616 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:29.616 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:29.616 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:29.616 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:29.616 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:29.616 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:29.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:29.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:29.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:29.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:29.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:29.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:29.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:29.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:29.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:29.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:29.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:29.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:29.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:29.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:29.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:29.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:29.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:29.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:29.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:29.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:29.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:29.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:29.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:29.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:29.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:29.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:29.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:29.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:29.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:29.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:29.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:29.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:29.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:29.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:29.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:29.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:29.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:29.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:29.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:29.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:29.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:29.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:29.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:29.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:29.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:29.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:29.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:29.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:29.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:29.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:29.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:29.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:29.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:29.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:29.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:29.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:29.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:29.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:29.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:29.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:29.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:29.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:29.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:29.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:29.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:29.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:29.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:29.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:29.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:29.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:29.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:29.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:29.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:29.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:29.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:29.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:29.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:29.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:29.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:29.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:29.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:29.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:29.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:29.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:29.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:29.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:29.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:29.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:29.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:29.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:29.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:29.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:29.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:29.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:29.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:29.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:29.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:29.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:29.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:29.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:29.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:29.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:29.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:29.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:30.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:30.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:30.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: pgmap v288: 161 pgs: 10 peering, 9 active+undersized+degraded, 13 active+undersized, 129 active+clean; 585 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s; 29/726 objects degraded (3.994%) 2024-06-01T02:11:30.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:30.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:30.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:30.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:30.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:30.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:30.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:30.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:30.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:30.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:30.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:30.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:30.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:30.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:30.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:30.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:30.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:30.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:30.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:30.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:30.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:30.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:30.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:30.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:30.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:30.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:30.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:30.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:30.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: pgmap v288: 161 pgs: 10 peering, 9 active+undersized+degraded, 13 active+undersized, 129 active+clean; 585 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s; 29/726 objects degraded (3.994%) 2024-06-01T02:11:30.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:30.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:30.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:30.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:30.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:30.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:30.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:30.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:30.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:30.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:30.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:30.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:30.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:30.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:30.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:30.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:30.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:30.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:30.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:30.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:30.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:30.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:30.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:30.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:30.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:30.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:30.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:30.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:30.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:30.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:30.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:30.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:30.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:30.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:30.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:30.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:30.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:30.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:30.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:30.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:30.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:30.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:30.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:30.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:30.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:30.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:30.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:30.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:30.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:30.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:30.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:30.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:30.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:30.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:30.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:30.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:30.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:30.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:30.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:30.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:30.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:30.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:30.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:30.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:30.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:30.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:30.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:30.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:30.801 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:30.801 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:30.801 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: pgmap v288: 161 pgs: 10 peering, 9 active+undersized+degraded, 13 active+undersized, 129 active+clean; 585 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s; 29/726 objects degraded (3.994%) 2024-06-01T02:11:30.802 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:30.802 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:30.802 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:30.802 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:30.802 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:30.802 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:30.802 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:30.802 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:30.802 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:30.802 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:30.802 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:30.802 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:30.802 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:30.803 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:30.803 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:30.803 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:30.803 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:30.803 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:30.803 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:30.803 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:30.803 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:30.803 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:30.803 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:30.803 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:30.803 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:30.803 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:30.803 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:30.803 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:30.804 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:30.804 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:30.804 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:30.804 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:30.804 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:30.804 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:30.804 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:30.804 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:30.804 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:30.804 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:30.804 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:30.804 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:30.804 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:30.804 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:30.804 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:30.805 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:30.805 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:30.805 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:30.805 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:31.776 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:31.776 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:31.776 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:31.776 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:31.776 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:31.776 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:31.776 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:31.777 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:31.777 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:31.777 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:31.777 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:31.777 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:31.777 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:31.777 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:31.777 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:31.777 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:31.777 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:31.777 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:31.777 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:31.777 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:31.778 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:31.778 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:31.778 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:31.778 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:31.778 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:31.778 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:31.778 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:31.778 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:31.778 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:31.778 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:11:31.778 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:31.778 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:31.778 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:31.779 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:31.779 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:31.779 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:31.779 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:31.779 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:31.779 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:31.779 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:31.779 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:31.779 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:31.779 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:31.779 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:31.779 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:31.780 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:31.780 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:31.780 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:31.780 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:31.780 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:31.780 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:31.780 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: Health check update: Degraded data redundancy: 29/726 objects degraded (3.994%), 9 pgs degraded (PG_DEGRADED) 2024-06-01T02:11:31.780 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:31.780 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:31.780 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:31.780 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:31.780 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:31.792 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:31.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:31.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:31.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:31.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:31.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:31.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:31.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:31.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:31.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:31.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:31.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:31.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:31.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:31.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:31.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:31.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:31.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:31.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:31.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:31.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:31.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:31.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:31.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:31.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:31.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:31.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:31.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:31.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:31.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:11:31.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:31.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:31.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:31.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:31.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:31.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:31.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:31.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:31.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:31.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:31.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:31.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:31.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:31.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:31.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:31.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:31.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:31.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:31.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:31.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:31.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:31.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:31.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:31.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:31.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:31.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:31.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:31.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:31.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:31.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:11:31.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:31.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:31.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:31.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:31.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:31.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:31.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:31.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:31.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:31.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:31.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:31.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:31.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:31.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:31.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:31.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:31.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:31.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:31.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:31.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:31.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:31.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: Health check update: Degraded data redundancy: 29/726 objects degraded (3.994%), 9 pgs degraded (PG_DEGRADED) 2024-06-01T02:11:31.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:31.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:31.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:31.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:31.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:31.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:31.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:31.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:31.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:31.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:31.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:31.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:31.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:31.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:31.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:31.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:31.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:31.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:31.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:31.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:31.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:31.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:31.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:31.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:31.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:31.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:31.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: Health check update: Degraded data redundancy: 29/726 objects degraded (3.994%), 9 pgs degraded (PG_DEGRADED) 2024-06-01T02:11:31.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:31.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:31.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:31.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:31.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:32.596 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:32.596 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: pgmap v289: 161 pgs: 10 peering, 151 active+clean; 585 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-06-01T02:11:32.596 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:32.596 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:32.596 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:32.596 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:32.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:32.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:32.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:32.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:32.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:32.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:32.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:32.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:32.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:32.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:32.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:32.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:32.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:32.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:32.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:32.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:32.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:32.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:32.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:32.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:32.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:32.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:32.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:32.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:32.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:32.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:32.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:32.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:32.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 29/726 objects degraded (3.994%), 9 pgs degraded) 2024-06-01T02:11:32.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: Cluster is now healthy 2024-06-01T02:11:32.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:32.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:32.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:32.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:32.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:32.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:32.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:32.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:32.600 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:32.600 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:32.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:32.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: pgmap v289: 161 pgs: 10 peering, 151 active+clean; 585 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-06-01T02:11:32.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:32.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:32.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:32.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:32.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:32.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:32.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:32.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:32.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:32.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:32.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:32.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:32.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:32.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:32.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:32.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:32.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:32.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:32.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:32.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:32.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:32.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:32.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:32.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:32.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:32.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:32.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:32.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:32.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: pgmap v289: 161 pgs: 10 peering, 151 active+clean; 585 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-06-01T02:11:32.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:32.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:32.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:32.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:32.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:32.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:32.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:32.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:32.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:32.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:32.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:32.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:32.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:32.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:32.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:32.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:32.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:32.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:32.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:32.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:32.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:32.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:32.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:32.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:32.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:32.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:32.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:32.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:32.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:32.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:32.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:32.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:32.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 29/726 objects degraded (3.994%), 9 pgs degraded) 2024-06-01T02:11:32.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: Cluster is now healthy 2024-06-01T02:11:32.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:32.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:32.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:32.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:32.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:32.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:32.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:32.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:32.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:32.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:32.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:32.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:32.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:32.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:32.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:32.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 29/726 objects degraded (3.994%), 9 pgs degraded) 2024-06-01T02:11:32.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: Cluster is now healthy 2024-06-01T02:11:32.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:32.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:32.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:32.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:32.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:32.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:32.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:32.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:32.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:32.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.526 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:33.526 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:11:33.526 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:33.526 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:33.526 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:33.527 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:33.527 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:33.527 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:33.778 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.778 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:33.778 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:33.779 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:33.779 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:33.779 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:33.779 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.779 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:33.779 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:33.779 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:33.779 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:33.779 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:33.779 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.779 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:33.779 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:33.779 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:33.779 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:33.780 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:33.780 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.780 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:33.780 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:33.780 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:33.780 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:33.780 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:33.780 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.780 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:33.780 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:33.780 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:33.780 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:33.780 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:33.780 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.781 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:33.781 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:33.781 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:33.781 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:33.781 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.781 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:33.781 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:33.781 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:33.781 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:33.781 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.781 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.781 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.781 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.781 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:33.782 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:33.782 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:33.782 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:33.782 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.792 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:33.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:11:33.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:33.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:33.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:33.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:33.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:33.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:33.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:33.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:33.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:33.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:33.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:33.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:33.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:33.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:33.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:33.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:33.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:33.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:33.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:33.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:33.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:33.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:33.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:33.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:33.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:11:33.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:33.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:33.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:33.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:33.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:33.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:33.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:33.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:33.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:33.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:33.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:33.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:33.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:33.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:33.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:33.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:33.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:33.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:33.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:33.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:33.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:33.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:33.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:33.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:33.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:33.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:33.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:33.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:33.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:33.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:33.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:33.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:33.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:33.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:33.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:33.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:33.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:33.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:33.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:33.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:33.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:33.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:33.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:33.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:33.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:33.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:33.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:33.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:33.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:33.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:33.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:33.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:33.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:33.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:33.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:33.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:33.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:33.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:33.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:33.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:33.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:33.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:33.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:33.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:33.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:34.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:34.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: pgmap v290: 161 pgs: 161 active+clean; 585 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-01T02:11:34.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:34.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:34.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:34.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:34.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:34.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:34.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:34.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:34.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:34.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:34.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:34.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:34.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:34.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:34.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:34.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:34.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:34.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:34.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:34.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:34.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:34.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:34.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:34.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:34.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:34.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:34.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:34.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:34.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:34.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:34.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:34.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:34.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:34.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:34.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:34.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:34.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:34.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:34.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:34.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:34.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:34.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:34.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:34.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:34.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:34.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:34.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:34.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:34.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: pgmap v290: 161 pgs: 161 active+clean; 585 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-01T02:11:34.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:34.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:34.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:34.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:34.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:34.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:34.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:34.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:34.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:34.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:34.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:34.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:34.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:34.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:34.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:34.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:34.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:34.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:34.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:34.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:34.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:34.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:34.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:34.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:34.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:34.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:34.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:34.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:34.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: pgmap v290: 161 pgs: 161 active+clean; 585 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-01T02:11:34.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:34.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:34.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:34.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:34.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:34.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:34.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:34.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:34.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:34.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:34.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:34.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:34.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:34.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:34.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:34.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:34.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:34.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:34.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:34.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:34.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:34.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:34.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:34.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:34.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:34.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:34.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:34.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:34.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:34.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:34.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:34.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:34.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:34.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:34.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:34.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:34.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:34.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:34.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:34.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:34.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:34.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:34.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:34.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:34.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:34.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:34.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:34.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:34.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:34.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:34.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:34.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:34.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:34.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:34.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:34.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:34.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:34.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:34.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:34.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:34.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:34.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:34.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:34.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:34.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:34.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:34.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:35.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:35.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:35.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:35.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:35.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:35.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:35.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:35.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:35.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:35.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:35.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:35.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:35.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:35.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:35.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:35.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:35.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:35.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:35.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:35.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:35.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:35.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:35.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:35.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:35.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:35.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:35.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:35.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:35.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:35.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:35.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:35.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:35.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:35.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:35.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:35.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:35.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:35.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:35.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:35.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:35.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:35.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:35.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:35.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:35.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:35.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:35.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:35.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:35.698 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:35.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:35.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:35.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:35.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:35.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:35.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:35.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:35.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:35.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:35.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:35.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:35.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:35.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:35.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:35.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:35.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:35.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:35.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:35.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:35.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:35.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:35.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:35.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:35.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:35.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:35.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:35.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:35.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:35.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:35.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:35.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:35.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:35.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:35.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:35.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:35.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:35.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:35.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:35.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:35.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:35.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:35.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:35.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:35.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:35.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:35.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:35.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:35.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:35.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:35.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:35.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:35.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:35.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:35.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:35.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:35.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:35.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:35.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:35.798 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:35.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:35.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:35.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:35.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:35.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:35.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:35.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:35.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:35.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:35.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:35.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:35.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:35.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:35.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:35.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:35.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:35.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:35.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:35.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:35.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:35.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:35.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:35.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:35.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:35.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:35.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:35.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:35.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:35.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:35.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:35.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:35.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:35.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:35.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:35.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:35.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:35.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:35.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:35.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:36.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-mon[84406]: pgmap v291: 161 pgs: 161 active+clean; 585 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-06-01T02:11:36.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:36.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:36.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:36.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:36.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:36.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:36.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:36.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:36.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:36.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[89502]: pgmap v291: 161 pgs: 161 active+clean; 585 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-06-01T02:11:36.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:36.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:36.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:36.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:36.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:36.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:36.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:36.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:36.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:36.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:36.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:36.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:36.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:36.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:36.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:36.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:36.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:36.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:36.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:36.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:36.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[100606]: pgmap v291: 161 pgs: 161 active+clean; 585 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-06-01T02:11:36.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:36.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:36.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:36.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:36.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:36.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:36.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:36.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:36.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:36.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:36.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:36.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:36.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:36.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:36.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:36.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:36.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:36.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:36.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:36.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:36.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:37.152 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:11:36 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:11:36] "GET /metrics HTTP/1.1" 200 38541 "" "Prometheus/2.33.4" 2024-06-01T02:11:37.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:37.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:37.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:37.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:37.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:37.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:37.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:37.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:37.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:37.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:37 smithi161 ceph-mon[84406]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-06-01T02:11:37.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-06-01T02:11:37.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:37 smithi161 ceph-mon[84406]: Upgrade: osd.7 is safe to restart 2024-06-01T02:11:38.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:38.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:38.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:38.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:38.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:38.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:38.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:38.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:38.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:38.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:37 smithi116 ceph-mon[89502]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-06-01T02:11:38.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-06-01T02:11:38.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:37 smithi116 ceph-mon[89502]: Upgrade: osd.7 is safe to restart 2024-06-01T02:11:38.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:38.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:38.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:38.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:38.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:38.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:38.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:38.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:38.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:38.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:37 smithi116 ceph-mon[100606]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-06-01T02:11:38.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-06-01T02:11:38.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:37 smithi116 ceph-mon[100606]: Upgrade: osd.7 is safe to restart 2024-06-01T02:11:38.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:38 smithi161 ceph-mon[84406]: pgmap v292: 161 pgs: 161 active+clean; 585 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-06-01T02:11:39.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:38 smithi116 ceph-mon[89502]: pgmap v292: 161 pgs: 161 active+clean; 585 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-06-01T02:11:39.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:38 smithi116 ceph-mon[100606]: pgmap v292: 161 pgs: 161 active+clean; 585 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-06-01T02:11:40.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:40 smithi161 ceph-mon[84406]: Upgrade: Updating osd.7 2024-06-01T02:11:40.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:40.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-01T02:11:40.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:40.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:40 smithi161 ceph-mon[84406]: Deploying daemon osd.7 on smithi161 2024-06-01T02:11:40.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:40 smithi161 ceph-mon[84406]: pgmap v293: 161 pgs: 161 active+clean; 585 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:11:40.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:40 smithi116 ceph-mon[89502]: Upgrade: Updating osd.7 2024-06-01T02:11:40.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:40.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-01T02:11:40.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:40.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:40 smithi116 ceph-mon[89502]: Deploying daemon osd.7 on smithi161 2024-06-01T02:11:40.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:40 smithi116 ceph-mon[89502]: pgmap v293: 161 pgs: 161 active+clean; 585 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:11:40.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:40 smithi116 ceph-mon[100606]: Upgrade: Updating osd.7 2024-06-01T02:11:40.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:40.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-01T02:11:40.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:40.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:40 smithi116 ceph-mon[100606]: Deploying daemon osd.7 on smithi161 2024-06-01T02:11:40.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:40 smithi116 ceph-mon[100606]: pgmap v293: 161 pgs: 161 active+clean; 585 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:11:41.166 INFO:teuthology.orchestra.run.smithi116.stdout:true 2024-06-01T02:11:41.824 INFO:teuthology.orchestra.run.smithi116.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-01T02:11:41.824 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi116 smithi116 running 8s ago 20m - - 2024-06-01T02:11:41.825 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi161 smithi161 running 5s ago 20m - - 2024-06-01T02:11:41.825 INFO:teuthology.orchestra.run.smithi116.stdout:alertmanager.a smithi116 *:9093,9094 running (9m) 8s ago 14m 22.2M - 0.23.0 ba2b418f427c e0c297162ded 2024-06-01T02:11:41.825 INFO:teuthology.orchestra.run.smithi116.stdout:grafana.a smithi161 *:3000 running (9m) 5s ago 14m 51.9M - 8.3.5 dad864ee21e9 00b0a78ee88e 2024-06-01T02:11:41.825 INFO:teuthology.orchestra.run.smithi116.stdout:iscsi.foo.smithi116.gjuxzb smithi116 running (8m) 8s ago 13m 62.9M - 3.5 e1d6a67b021e 0e191fe3e3bb 2024-06-01T02:11:41.825 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.x smithi161 *:8443,9283,8765 running (8m) 5s ago 19m 537M - 19.0.0-2946-g3d89accf b045ac545f82 2d7fbb85e5f7 2024-06-01T02:11:41.825 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.y smithi116 *:8443,9283,8765 running (7m) 8s ago 21m 456M - 19.0.0-2946-g3d89accf b045ac545f82 bc9627c27a57 2024-06-01T02:11:41.825 INFO:teuthology.orchestra.run.smithi116.stdout:mon.a smithi116 running (6m) 8s ago 21m 73.5M 2048M 19.0.0-2946-g3d89accf b045ac545f82 b5e9b6f873ff 2024-06-01T02:11:41.825 INFO:teuthology.orchestra.run.smithi116.stdout:mon.b smithi161 running (5m) 5s ago 19m 49.9M 2048M 19.0.0-2946-g3d89accf b045ac545f82 aaa08c03cfc8 2024-06-01T02:11:41.825 INFO:teuthology.orchestra.run.smithi116.stdout:mon.c smithi116 running (6m) 8s ago 19m 57.0M 2048M 19.0.0-2946-g3d89accf b045ac545f82 bd0fb0e603fa 2024-06-01T02:11:41.825 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.a smithi116 *:9100 running (9m) 8s ago 15m 17.4M - 1.3.1 1dbe0e931976 e347748d17bb 2024-06-01T02:11:41.825 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.b smithi161 *:9100 running (9m) 5s ago 15m 20.3M - 1.3.1 1dbe0e931976 6e66182f5a1a 2024-06-01T02:11:41.825 INFO:teuthology.orchestra.run.smithi116.stdout:osd.0 smithi116 running (4m) 8s ago 19m 53.4M 4096M 19.0.0-2946-g3d89accf b045ac545f82 b4313d121b9d 2024-06-01T02:11:41.825 INFO:teuthology.orchestra.run.smithi116.stdout:osd.1 smithi116 running (4m) 8s ago 18m 56.5M 4096M 19.0.0-2946-g3d89accf b045ac545f82 dccdbeac5aae 2024-06-01T02:11:41.826 INFO:teuthology.orchestra.run.smithi116.stdout:osd.2 smithi116 running (3m) 8s ago 18m 49.5M 4096M 19.0.0-2946-g3d89accf b045ac545f82 1ad95efda404 2024-06-01T02:11:41.826 INFO:teuthology.orchestra.run.smithi116.stdout:osd.3 smithi116 running (2m) 8s ago 17m 52.9M 4096M 19.0.0-2946-g3d89accf b045ac545f82 bd00ad2c2556 2024-06-01T02:11:41.826 INFO:teuthology.orchestra.run.smithi116.stdout:osd.4 smithi161 running (2m) 5s ago 17m 52.0M 4096M 19.0.0-2946-g3d89accf b045ac545f82 0084b9edf7a0 2024-06-01T02:11:41.826 INFO:teuthology.orchestra.run.smithi116.stdout:osd.5 smithi161 running (60s) 5s ago 17m 48.7M 4096M 19.0.0-2946-g3d89accf b045ac545f82 b3849cdeb707 2024-06-01T02:11:41.826 INFO:teuthology.orchestra.run.smithi116.stdout:osd.6 smithi161 running (21s) 5s ago 16m 43.4M 4096M 19.0.0-2946-g3d89accf b045ac545f82 6b87d9299c8d 2024-06-01T02:11:41.826 INFO:teuthology.orchestra.run.smithi116.stdout:osd.7 smithi161 running (16m) 5s ago 16m 66.1M 4096M 17.2.0 e1d6a67b021e 6004e6983735 2024-06-01T02:11:41.826 INFO:teuthology.orchestra.run.smithi116.stdout:prometheus.a smithi161 *:9095 running (8m) 5s ago 15m 79.0M - 2.33.4 514e6a882f6e e977f6ca1d73 2024-06-01T02:11:41.826 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi116.vzjrxj smithi116 *:8000 running (13m) 8s ago 13m 78.0M - 17.2.0 e1d6a67b021e df4fe3c1dbc3 2024-06-01T02:11:41.826 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi161.ashqxz smithi161 *:8000 running (13m) 5s ago 13m 78.5M - 17.2.0 e1d6a67b021e 3eecc1f3e2d3 2024-06-01T02:11:41.826 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi116.hokbmf smithi116 *:80 running (13m) 8s ago 13m 78.3M - 17.2.0 e1d6a67b021e 10950ed75dbd 2024-06-01T02:11:41.826 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi161.axjlzw smithi161 *:80 running (13m) 5s ago 13m 77.7M - 17.2.0 e1d6a67b021e 0b5d5ce47772 2024-06-01T02:11:42.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:42.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:42 smithi161 ceph-mon[84406]: from='client.44589 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:11:42.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:42 smithi161 ceph-mon[84406]: pgmap v294: 161 pgs: 161 active+clean; 585 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:11:42.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:42 smithi161 ceph-mon[84406]: from='client.44595 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:11:42.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:42 smithi161 ceph-mon[84406]: from='client.44601 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:11:42.228 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:11:42.228 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": { 2024-06-01T02:11:42.228 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 3 2024-06-01T02:11:42.228 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:11:42.229 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": { 2024-06-01T02:11:42.229 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-01T02:11:42.229 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:11:42.229 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": { 2024-06-01T02:11:42.229 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 1, 2024-06-01T02:11:42.229 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 7 2024-06-01T02:11:42.229 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:11:42.229 INFO:teuthology.orchestra.run.smithi116.stdout: "rgw": { 2024-06-01T02:11:42.229 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-06-01T02:11:42.229 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:11:42.229 INFO:teuthology.orchestra.run.smithi116.stdout: "overall": { 2024-06-01T02:11:42.229 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 5, 2024-06-01T02:11:42.229 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 12 2024-06-01T02:11:42.229 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T02:11:42.230 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:11:42.230 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:42.230 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:42 smithi116 ceph-mon[100606]: from='client.44589 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:11:42.230 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:42 smithi116 ceph-mon[100606]: pgmap v294: 161 pgs: 161 active+clean; 585 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:11:42.230 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:42 smithi116 ceph-mon[100606]: from='client.44595 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:11:42.230 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:42 smithi116 ceph-mon[100606]: from='client.44601 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:11:42.230 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:42.230 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:42 smithi116 ceph-mon[89502]: from='client.44589 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:11:42.231 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:42 smithi116 ceph-mon[89502]: pgmap v294: 161 pgs: 161 active+clean; 585 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:11:42.231 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:42 smithi116 ceph-mon[89502]: from='client.44595 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:11:42.231 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:42 smithi116 ceph-mon[89502]: from='client.44601 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:11:42.565 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:11:42.565 INFO:teuthology.orchestra.run.smithi116.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-01T02:11:42.565 INFO:teuthology.orchestra.run.smithi116.stdout: "in_progress": true, 2024-06-01T02:11:42.565 INFO:teuthology.orchestra.run.smithi116.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-01T02:11:42.565 INFO:teuthology.orchestra.run.smithi116.stdout: "services_complete": [ 2024-06-01T02:11:42.565 INFO:teuthology.orchestra.run.smithi116.stdout: "mon", 2024-06-01T02:11:42.565 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr" 2024-06-01T02:11:42.565 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-06-01T02:11:42.566 INFO:teuthology.orchestra.run.smithi116.stdout: "progress": "12/23 daemons upgraded", 2024-06-01T02:11:42.566 INFO:teuthology.orchestra.run.smithi116.stdout: "message": "Currently upgrading osd daemons", 2024-06-01T02:11:42.566 INFO:teuthology.orchestra.run.smithi116.stdout: "is_paused": false 2024-06-01T02:11:42.566 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:11:42.972 INFO:teuthology.orchestra.run.smithi116.stdout:HEALTH_OK 2024-06-01T02:11:43.204 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:43 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:11:43.205 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:43 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/391257911' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:43.205 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:43 smithi116 ceph-mon[100606]: from='client.64576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:11:43.205 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:43 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/1180809434' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:11:43.205 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:43 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:11:43.205 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:43 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/391257911' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:43.205 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:43 smithi116 ceph-mon[89502]: from='client.64576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:11:43.205 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:43 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/1180809434' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:11:43.378 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:43 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:11:43.378 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:43 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/391257911' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:43.378 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:43 smithi161 ceph-mon[84406]: from='client.64576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:11:43.378 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:43 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/1180809434' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:11:44.242 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:44 smithi116 ceph-mon[89502]: pgmap v295: 161 pgs: 161 active+clean; 585 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:11:44.242 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:44 smithi116 ceph-mon[100606]: pgmap v295: 161 pgs: 161 active+clean; 585 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:11:44.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:44 smithi161 ceph-mon[84406]: pgmap v295: 161 pgs: 161 active+clean; 585 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:11:44.694 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:44 smithi161 systemd[1]: Stopping Ceph osd.7 for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:11:45.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:45 smithi161 ceph-mon[84406]: osd.7 marked itself down and dead 2024-06-01T02:11:45.194 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:44 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[39465]: 2024-06-01T02:11:44.798+0000 7fc5c7f62700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-01T02:11:45.194 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:44 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[39465]: 2024-06-01T02:11:44.798+0000 7fc5c7f62700 -1 osd.7 185 *** Got signal Terminated *** 2024-06-01T02:11:45.194 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:44 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[39465]: 2024-06-01T02:11:44.798+0000 7fc5c7f62700 -1 osd.7 185 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-01T02:11:45.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:45 smithi116 ceph-mon[89502]: osd.7 marked itself down and dead 2024-06-01T02:11:45.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:45 smithi116 ceph-mon[100606]: osd.7 marked itself down and dead 2024-06-01T02:11:45.694 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:45 smithi161 podman[126305]: 2024-06-01 02:11:45.432809703 +0000 UTC m=+0.731331721 container died 6004e69837357c6264cc6f5b37969906eaf68b4d794d0b606c2db85e0256c15d (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7, name=centos-stream, GIT_CLEAN=True, vcs-type=git, com.redhat.component=centos-stream-container, distribution-scope=public, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, io.buildah.version=1.19.8, CEPH_POINT_RELEASE=-17.2.0, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, io.openshift.expose-services=, architecture=x86_64, build-date=2022-05-03T08:36:31.336870, RELEASE=HEAD, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, version=8, release=754, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.tags=base centos centos-stream, io.k8s.display-name=CentOS Stream 8, GIT_BRANCH=HEAD, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, ceph=True, vendor=Red Hat, Inc.) 2024-06-01T02:11:46.021 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:45 smithi161 podman[126305]: 2024-06-01 02:11:45.832664112 +0000 UTC m=+1.131186130 container cleanup 6004e69837357c6264cc6f5b37969906eaf68b4d794d0b606c2db85e0256c15d (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7, io.openshift.expose-services=, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, CEPH_POINT_RELEASE=-17.2.0, version=8, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, distribution-scope=public, com.redhat.component=centos-stream-container, io.buildah.version=1.19.8, build-date=2022-05-03T08:36:31.336870, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, name=centos-stream, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, io.openshift.tags=base centos centos-stream, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, vendor=Red Hat, Inc., architecture=x86_64, GIT_CLEAN=True, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, io.k8s.display-name=CentOS Stream 8, vcs-type=git, RELEASE=HEAD, release=754, ceph=True, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, GIT_BRANCH=HEAD) 2024-06-01T02:11:46.021 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:45 smithi161 bash[126305]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7 2024-06-01T02:11:46.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:46 smithi116 ceph-mon[89502]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:11:46.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:46 smithi116 ceph-mon[89502]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-06-01T02:11:46.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:46 smithi116 ceph-mon[89502]: osdmap e186: 8 total, 7 up, 8 in 2024-06-01T02:11:46.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:46 smithi116 ceph-mon[89502]: pgmap v297: 161 pgs: 20 stale+active+clean, 141 active+clean; 585 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:11:46.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:46.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:11:46.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:46 smithi116 ceph-mon[100606]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:11:46.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:46 smithi116 ceph-mon[100606]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-06-01T02:11:46.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:46 smithi116 ceph-mon[100606]: osdmap e186: 8 total, 7 up, 8 in 2024-06-01T02:11:46.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:46 smithi116 ceph-mon[100606]: pgmap v297: 161 pgs: 20 stale+active+clean, 141 active+clean; 585 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:11:46.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:46.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:11:46.438 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:46 smithi161 podman[126322]: 2024-06-01 02:11:46.025930696 +0000 UTC m=+0.587019008 container remove 6004e69837357c6264cc6f5b37969906eaf68b4d794d0b606c2db85e0256c15d (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7, GIT_BRANCH=HEAD, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, io.openshift.expose-services=, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, build-date=2022-05-03T08:36:31.336870, GIT_REPO=https://github.com/ceph/ceph-container.git, vcs-type=git, architecture=x86_64, io.openshift.tags=base centos centos-stream, maintainer=Guillaume Abrioux , version=8, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, name=centos-stream, distribution-scope=public, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, vendor=Red Hat, Inc., RELEASE=HEAD, com.redhat.component=centos-stream-container, io.buildah.version=1.19.8, ceph=True, GIT_CLEAN=True, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, CEPH_POINT_RELEASE=-17.2.0, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, release=754, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.k8s.display-name=CentOS Stream 8, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream) 2024-06-01T02:11:46.438 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:46 smithi161 ceph-mon[84406]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:11:46.438 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:46 smithi161 ceph-mon[84406]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-06-01T02:11:46.438 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:46 smithi161 ceph-mon[84406]: osdmap e186: 8 total, 7 up, 8 in 2024-06-01T02:11:46.439 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:46 smithi161 ceph-mon[84406]: pgmap v297: 161 pgs: 20 stale+active+clean, 141 active+clean; 585 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:11:46.439 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:46.439 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:11:46.694 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:46 smithi161 podman[126413]: 2024-06-01 02:11:46.443428218 +0000 UTC m=+0.093576857 container create 8ab312991eca6c6bf942ecc199eff4cde7daa50a2d26b7905919bc568b492ef7 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240528) 2024-06-01T02:11:46.694 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:46 smithi161 podman[126413]: 2024-06-01 02:11:46.367706392 +0000 UTC m=+0.017855051 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:11:46.694 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:46 smithi161 podman[126413]: 2024-06-01 02:11:46.662950374 +0000 UTC m=+0.313099015 container init 8ab312991eca6c6bf942ecc199eff4cde7daa50a2d26b7905919bc568b492ef7 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-01T02:11:46.694 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:46 smithi161 podman[126413]: 2024-06-01 02:11:46.666370487 +0000 UTC m=+0.316519132 container start 8ab312991eca6c6bf942ecc199eff4cde7daa50a2d26b7905919bc568b492ef7 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-01T02:11:47.047 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:11:46 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:11:46] "GET /metrics HTTP/1.1" 200 38618 "" "Prometheus/2.33.4" 2024-06-01T02:11:47.048 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:46 smithi161 podman[126413]: 2024-06-01 02:11:46.771423491 +0000 UTC m=+0.421572179 container attach 8ab312991eca6c6bf942ecc199eff4cde7daa50a2d26b7905919bc568b492ef7 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-06-01T02:11:47.048 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:46 smithi161 podman[126413]: 2024-06-01 02:11:46.961344966 +0000 UTC m=+0.611493620 container died 8ab312991eca6c6bf942ecc199eff4cde7daa50a2d26b7905919bc568b492ef7 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-06-01T02:11:47.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:47 smithi161 ceph-mon[84406]: osdmap e187: 8 total, 7 up, 8 in 2024-06-01T02:11:47.503 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:47 smithi116 ceph-mon[89502]: osdmap e187: 8 total, 7 up, 8 in 2024-06-01T02:11:47.503 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:47 smithi116 ceph-mon[100606]: osdmap e187: 8 total, 7 up, 8 in 2024-06-01T02:11:47.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:47 smithi161 podman[126439]: 2024-06-01 02:11:47.550454497 +0000 UTC m=+0.583263363 container remove 8ab312991eca6c6bf942ecc199eff4cde7daa50a2d26b7905919bc568b492ef7 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-deactivate, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-06-01T02:11:47.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:47 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.7.service: Deactivated successfully. 2024-06-01T02:11:47.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:47 smithi161 systemd[1]: Stopped Ceph osd.7 for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:11:47.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:47 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.7.service: Consumed 14.914s CPU time. 2024-06-01T02:11:47.945 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:47 smithi161 systemd[1]: Starting Ceph osd.7 for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:11:48.241 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:47 smithi161 podman[126538]: 2024-06-01 02:11:47.993227652 +0000 UTC m=+0.109950199 container create d2b6cd1a0525c6bf77589a94059a8d93385d53ca2485a6ce24e9cedb2d0fc6b1 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-activate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-06-01T02:11:48.241 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:47 smithi161 podman[126538]: 2024-06-01 02:11:47.900374743 +0000 UTC m=+0.017097253 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:11:48.241 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:48 smithi161 ceph-mon[84406]: pgmap v299: 161 pgs: 1 active+undersized+degraded, 7 active+undersized, 20 peering, 8 stale+active+clean, 125 active+clean; 585 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 5/726 objects degraded (0.689%) 2024-06-01T02:11:48.542 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:48 smithi116 ceph-mon[89502]: pgmap v299: 161 pgs: 1 active+undersized+degraded, 7 active+undersized, 20 peering, 8 stale+active+clean, 125 active+clean; 585 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 5/726 objects degraded (0.689%) 2024-06-01T02:11:48.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:48 smithi116 ceph-mon[100606]: pgmap v299: 161 pgs: 1 active+undersized+degraded, 7 active+undersized, 20 peering, 8 stale+active+clean, 125 active+clean; 585 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 5/726 objects degraded (0.689%) 2024-06-01T02:11:48.694 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:48 smithi161 podman[126538]: 2024-06-01 02:11:48.246159454 +0000 UTC m=+0.362881968 container init d2b6cd1a0525c6bf77589a94059a8d93385d53ca2485a6ce24e9cedb2d0fc6b1 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-activate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_CLEAN=True, RELEASE=squid-3d89acc, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-06-01T02:11:48.694 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:48 smithi161 podman[126538]: 2024-06-01 02:11:48.25015291 +0000 UTC m=+0.366875412 container start d2b6cd1a0525c6bf77589a94059a8d93385d53ca2485a6ce24e9cedb2d0fc6b1 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-01T02:11:48.694 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:48 smithi161 podman[126538]: 2024-06-01 02:11:48.324765477 +0000 UTC m=+0.441487981 container attach d2b6cd1a0525c6bf77589a94059a8d93385d53ca2485a6ce24e9cedb2d0fc6b1 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-activate, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-06-01T02:11:49.055 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:48 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-activate[126554]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-06-01T02:11:49.056 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:48 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-activate[126554]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-7 --no-mon-config --dev /dev/mapper/vg_nvme-lv_1 2024-06-01T02:11:49.056 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:48 smithi161 bash[126538]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-06-01T02:11:49.056 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:48 smithi161 bash[126538]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-7 --no-mon-config --dev /dev/mapper/vg_nvme-lv_1 2024-06-01T02:11:49.056 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:48 smithi161 bash[126538]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_1 2024-06-01T02:11:49.056 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:48 smithi161 bash[126538]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0 2024-06-01T02:11:49.056 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:48 smithi161 bash[126538]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-7/block 2024-06-01T02:11:49.056 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:48 smithi161 bash[126538]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-06-01T02:11:49.056 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:48 smithi161 bash[126538]: --> ceph-volume raw activate successful for osd ID: 7 2024-06-01T02:11:49.056 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:48 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-activate[126554]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_1 2024-06-01T02:11:49.056 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:48 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-activate[126554]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0 2024-06-01T02:11:49.056 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:48 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-activate[126554]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-7/block 2024-06-01T02:11:49.056 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:48 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-activate[126554]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-06-01T02:11:49.057 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:48 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-activate[126554]: --> ceph-volume raw activate successful for osd ID: 7 2024-06-01T02:11:49.057 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:48 smithi161 podman[126538]: 2024-06-01 02:11:48.725675365 +0000 UTC m=+0.842397867 container died d2b6cd1a0525c6bf77589a94059a8d93385d53ca2485a6ce24e9cedb2d0fc6b1 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-activate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-06-01T02:11:49.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:49 smithi116 ceph-mon[100606]: Health check failed: Reduced data availability: 2 pgs inactive, 4 pgs peering (PG_AVAILABILITY) 2024-06-01T02:11:49.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:49 smithi116 ceph-mon[100606]: Health check failed: Degraded data redundancy: 5/726 objects degraded (0.689%), 1 pg degraded (PG_DEGRADED) 2024-06-01T02:11:49.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:49 smithi116 ceph-mon[89502]: Health check failed: Reduced data availability: 2 pgs inactive, 4 pgs peering (PG_AVAILABILITY) 2024-06-01T02:11:49.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:49 smithi116 ceph-mon[89502]: Health check failed: Degraded data redundancy: 5/726 objects degraded (0.689%), 1 pg degraded (PG_DEGRADED) 2024-06-01T02:11:49.386 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:49 smithi161 ceph-mon[84406]: Health check failed: Reduced data availability: 2 pgs inactive, 4 pgs peering (PG_AVAILABILITY) 2024-06-01T02:11:49.386 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:49 smithi161 ceph-mon[84406]: Health check failed: Degraded data redundancy: 5/726 objects degraded (0.689%), 1 pg degraded (PG_DEGRADED) 2024-06-01T02:11:49.694 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:49 smithi161 podman[126681]: 2024-06-01 02:11:49.391570338 +0000 UTC m=+0.660209766 container remove d2b6cd1a0525c6bf77589a94059a8d93385d53ca2485a6ce24e9cedb2d0fc6b1 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-activate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-06-01T02:11:49.694 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:49 smithi161 podman[126740]: 2024-06-01 02:11:49.509333143 +0000 UTC m=+0.017720313 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:11:49.694 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:49 smithi161 podman[126740]: 2024-06-01 02:11:49.634215368 +0000 UTC m=+0.142602527 container create c98098e14cc9373f55402d74ae0d4184930f1acf5db306131fb434c838c87ad3 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528) 2024-06-01T02:11:50.194 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:49 smithi161 podman[126740]: 2024-06-01 02:11:49.828156341 +0000 UTC m=+0.336543507 container init c98098e14cc9373f55402d74ae0d4184930f1acf5db306131fb434c838c87ad3 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-06-01T02:11:50.194 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:49 smithi161 podman[126740]: 2024-06-01 02:11:49.831411225 +0000 UTC m=+0.339798374 container start c98098e14cc9373f55402d74ae0d4184930f1acf5db306131fb434c838c87ad3 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528) 2024-06-01T02:11:50.194 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:49 smithi161 ceph-osd[126759]: -- 172.21.15.161:0/1024809994 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563ba3167680 con 0x563ba3131400 2024-06-01T02:11:50.194 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:49 smithi161 bash[126740]: c98098e14cc9373f55402d74ae0d4184930f1acf5db306131fb434c838c87ad3 2024-06-01T02:11:50.194 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:49 smithi161 systemd[1]: Started Ceph osd.7 for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:11:50.694 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[126755]: 2024-06-01T02:11:50.379+0000 7f3bffea17c0 -1 Falling back to public interface 2024-06-01T02:11:51.217 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: pgmap v300: 161 pgs: 18 active+undersized+degraded, 24 active+undersized, 20 peering, 99 active+clean; 585 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 73/726 objects degraded (10.055%) 2024-06-01T02:11:51.217 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:51.217 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:51.218 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:51.218 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:51.218 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:51.218 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:51.218 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:51.218 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:51.218 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:51.218 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:51.218 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:51.218 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:51.218 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:51.218 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:51.219 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:51.219 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:51.219 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:51.219 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:51.219 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:51.219 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:51.219 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:51.219 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:51.219 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:51.219 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:51.219 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:51.219 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:51.219 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:51.220 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:51.220 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:51.220 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:51.220 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:51.220 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:51.220 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:51.220 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:51.220 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:51.220 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:51.220 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:51.220 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:51.220 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:51.220 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:50 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:51.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: pgmap v300: 161 pgs: 18 active+undersized+degraded, 24 active+undersized, 20 peering, 99 active+clean; 585 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 73/726 objects degraded (10.055%) 2024-06-01T02:11:51.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:51.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:51.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:51.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:51.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:51.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:51.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:51.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:51.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:51.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:51.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:51.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:51.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:51.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:51.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:51.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:51.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:51.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:51.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:51.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:51.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:51.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:51.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:51.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:51.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:51.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:51.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:51.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:51.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:51.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:51.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:51.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:51.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:51.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:51.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:51.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:51.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:51.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:51.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:51.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:51.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: pgmap v300: 161 pgs: 18 active+undersized+degraded, 24 active+undersized, 20 peering, 99 active+clean; 585 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 73/726 objects degraded (10.055%) 2024-06-01T02:11:51.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:51.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:51.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:51.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:51.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:51.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:51.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:51.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:51.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:51.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:51.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:51.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:51.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:51.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:51.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:51.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:51.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:51.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:51.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:51.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:51.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:51.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:51.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:51.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:51.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:51.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:51.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:51.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:51.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:51.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:51.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:51.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:51.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:51.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:51.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:51.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:51.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:51.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:51.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:51.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:50 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:52.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:52.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:52.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:52.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:52.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:52.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:52.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:52.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:52.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:52.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:52.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:52.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:52.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:52.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:52.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:52.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:52.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:52.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:52.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:52.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:52.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:52.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:52.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:52.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:52.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:52.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:52.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:52.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:52.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:52.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:52.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:52.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:52.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:52.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:52.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:52.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:52.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:52.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:52.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:52.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:52.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:52.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:52.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:52.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:52.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:52.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:52.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:52.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:52.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:52.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:52.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:52.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:52.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:52.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:52.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:52.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:52.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:52.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:52.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:52.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:52.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:52.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:52.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:52.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:52.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:52.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:52.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:52.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:52.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:52.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:52.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:52.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:52.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:52.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:52.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:52.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:52.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:52.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:52.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:52.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:52.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:52.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:52.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:52.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:52.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:52.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:52.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:52.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:52.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:52.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:52.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:52.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:52.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:52.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:52.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:52.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:52.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:52.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:52.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:52.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:52.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:52.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:52.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:52.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:52.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:52.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:52.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:52.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:52.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:52.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:52.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:52.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:52.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:52.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:52.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:52.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:52.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:52.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:52.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:52.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:52.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:52.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:52.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:52.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:52.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:52.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:52.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:52.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:52.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:52.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:52.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:52.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:52.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:52.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:52.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:52.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:52.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:52.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:52.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:52.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:52.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:52.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:52.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:52.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:52.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:52.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:52.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:52.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:52.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:52.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:52.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:52.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:52.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:52.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:52.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:52.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:52.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:52.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:52.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:53.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:53.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:53.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: pgmap v301: 161 pgs: 24 active+undersized+degraded, 38 active+undersized, 99 active+clean; 585 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 91/726 objects degraded (12.534%) 2024-06-01T02:11:53.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:53.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:53.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:53.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:53.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:53.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:53.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:53.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:53.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:53.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:53.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:53.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:53.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:53.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:53.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:53.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:53.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:53.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 4 pgs peering) 2024-06-01T02:11:53.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:53.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:53.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:53.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:53.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:53.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:53.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:53.105 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:53.105 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:53.105 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:53.105 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:53.105 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:53.105 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:53.105 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:53.105 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:53.105 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:53.105 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:53.105 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:53.105 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:53.106 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:53.106 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:53.106 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:53.106 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:53.106 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:53.106 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:53.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:53.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:53.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: pgmap v301: 161 pgs: 24 active+undersized+degraded, 38 active+undersized, 99 active+clean; 585 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 91/726 objects degraded (12.534%) 2024-06-01T02:11:53.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:53.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:53.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:53.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:53.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:53.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:53.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:53.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:53.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:53.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:53.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:53.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:53.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:53.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:53.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:53.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:53.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:53.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 4 pgs peering) 2024-06-01T02:11:53.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:53.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:53.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:53.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:53.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:53.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:53.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:53.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:53.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:53.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:53.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: pgmap v301: 161 pgs: 24 active+undersized+degraded, 38 active+undersized, 99 active+clean; 585 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 91/726 objects degraded (12.534%) 2024-06-01T02:11:53.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:53.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:53.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:53.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:53.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:53.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:53.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:53.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:53.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:53.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:53.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:53.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:53.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:53.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:53.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:53.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:53.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:53.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 4 pgs peering) 2024-06-01T02:11:53.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:53.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:53.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:53.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:53.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:53.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:53.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:53.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:53.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:53.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:53.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:53.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:53.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:53.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:53.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:53.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:53.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:53.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:53.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:53.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:53.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:53.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:53.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:53.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:53.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:53.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:53.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:53.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:53.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:53.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:53.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:53.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:53.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:53.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:53.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:53.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:53.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:53.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:53.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:53.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:53.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:53.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:53.364 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[126755]: 2024-06-01T02:11:53.106+0000 7f3bffea17c0 -1 osd.7 0 read_superblock omap replica is missing. 2024-06-01T02:11:53.364 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[126755]: 2024-06-01T02:11:53.158+0000 7f3bffea17c0 -1 osd.7 185 log_to_monitors true 2024-06-01T02:11:54.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:54.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:11:54.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:54.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:54.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:54.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:54.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:54.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:54.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:54.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:54.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:54.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:54.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:54.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='osd.7 [v2:172.21.15.161:6824/319989768,v1:172.21.15.161:6825/319989768]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-01T02:11:54.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-01T02:11:54.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:54.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:54.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:54.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:54.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:54.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:54.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:54.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:54.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:54.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:54.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:54.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:54.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:54.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:54.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:54.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:54.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:54.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:54.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:54.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:54.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:54.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:54.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:54.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:54.097 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:54.097 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:54.097 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:54.097 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:54.097 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:54.097 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:54.097 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:54.097 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:54.097 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:53 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:54.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:54.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:11:54.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:54.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:54.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:54.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:54.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:54.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:54.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:54.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:54.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:54.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:54.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:54.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='osd.7 [v2:172.21.15.161:6824/319989768,v1:172.21.15.161:6825/319989768]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-01T02:11:54.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-01T02:11:54.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:54.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:54.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:54.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:54.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:54.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:54.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:54.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:54.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:54.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:54.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:54.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:54.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:54.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:54.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:11:54.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:54.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:54.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:54.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:54.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:54.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:54.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:54.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:54.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:54.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:54.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:54.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='osd.7 [v2:172.21.15.161:6824/319989768,v1:172.21.15.161:6825/319989768]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-01T02:11:54.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-01T02:11:54.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:54.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:54.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:54.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:54.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:54.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:54.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:54.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:54.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:54.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:54.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:54.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:54.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:54.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:54.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:54.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:54.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:54.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:54.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:54.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:54.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:54.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:54.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:54.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:54.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:54.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:54.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:54.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:54.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:54.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:54.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:54.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:54.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:54.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:54.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:54.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:54.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:54.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:54.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:54.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:54.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:54.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:54.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:54.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:54.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:54.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:54.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:54.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:54.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:54.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:54.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:54.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:54.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:53 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:54.415 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[126755]: 2024-06-01T02:11:54.096+0000 7f3bf12f8700 -1 osd.7 185 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-01T02:11:55.079 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:55.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: pgmap v302: 161 pgs: 24 active+undersized+degraded, 38 active+undersized, 99 active+clean; 585 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 91/726 objects degraded (12.534%) 2024-06-01T02:11:55.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:55.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:55.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:55.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:55.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:55.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:55.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-01T02:11:55.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='osd.7 [v2:172.21.15.161:6824/319989768,v1:172.21.15.161:6825/319989768]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:11:55.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: osdmap e188: 8 total, 7 up, 8 in 2024-06-01T02:11:55.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:11:55.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:55.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:55.081 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:55.081 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:55.081 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:55.081 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:55.081 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:55.081 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:55.081 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:55.081 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:55.081 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:55.081 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:55.081 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:55.081 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:55.081 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:55.081 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:55.081 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:55.082 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:55.082 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:55.082 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:55.082 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:55.082 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:55.082 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:55.082 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:55.082 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:55.082 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:55.082 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:55.082 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:55.082 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:55.082 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:55.082 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:55.083 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:55.083 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:55.083 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:55.083 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:55.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:55.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: pgmap v302: 161 pgs: 24 active+undersized+degraded, 38 active+undersized, 99 active+clean; 585 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 91/726 objects degraded (12.534%) 2024-06-01T02:11:55.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:55.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:55.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:55.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:55.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:55.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:55.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-01T02:11:55.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='osd.7 [v2:172.21.15.161:6824/319989768,v1:172.21.15.161:6825/319989768]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:11:55.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: osdmap e188: 8 total, 7 up, 8 in 2024-06-01T02:11:55.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:11:55.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:55.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:55.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:55.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:55.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:55.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:55.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:55.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:55.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:55.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:55.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:55.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:55.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:55.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:55.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:55.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:55.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:55.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: pgmap v302: 161 pgs: 24 active+undersized+degraded, 38 active+undersized, 99 active+clean; 585 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 91/726 objects degraded (12.534%) 2024-06-01T02:11:55.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:55.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:55.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:55.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:55.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:55.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:55.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-01T02:11:55.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='osd.7 [v2:172.21.15.161:6824/319989768,v1:172.21.15.161:6825/319989768]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:11:55.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: osdmap e188: 8 total, 7 up, 8 in 2024-06-01T02:11:55.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:11:55.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:55.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:55.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:55.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:55.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:55.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:55.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:55.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:55.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:55.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:55.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:55.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:55.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:55.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:55.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:55.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:55.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:55.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:55.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:55.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:55.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:55.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:55.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:55.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:55.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:55.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:55.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:55.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:55.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:55.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:55.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:55.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:55.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:55.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:55.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:55.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:55.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:55.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:55.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:55.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:55.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:55.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:55.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:55.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:55.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:55.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:55.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:55.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:55.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:55.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:55.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:55.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:55.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:55.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:56.240 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:56.240 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:56.240 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:56.240 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:56.240 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:56.240 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:56.240 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:56.240 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:56.240 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:56.241 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:56.241 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:56.241 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:56.241 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:11:56.241 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: osd.7 [v2:172.21.15.161:6824/319989768,v1:172.21.15.161:6825/319989768] boot 2024-06-01T02:11:56.241 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: osdmap e189: 8 total, 8 up, 8 in 2024-06-01T02:11:56.241 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T02:11:56.241 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:56.241 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:56.241 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:56.241 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:56.241 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:56.241 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:56.241 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:56.241 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:56.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:56.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:56.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:56.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:56.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:56.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:56.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:56.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:56.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:56.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:56.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:56.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:56.244 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:56.244 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:56.244 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:56.244 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:56.244 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:56.244 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:56.244 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:56.244 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:56.244 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:56.244 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:56.245 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:56.245 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:56.245 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:56.245 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:56.245 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:56.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:56.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:56.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:56.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:56.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:56.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:56.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:56.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:56.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:56.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:56.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:56.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:56.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:11:56.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: osd.7 [v2:172.21.15.161:6824/319989768,v1:172.21.15.161:6825/319989768] boot 2024-06-01T02:11:56.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: osdmap e189: 8 total, 8 up, 8 in 2024-06-01T02:11:56.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T02:11:56.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:56.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:56.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:56.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:56.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:56.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:56.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:56.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:56.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:56.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:56.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:56.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:56.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:56.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:56.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:56.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:56.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:56.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:56.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:56.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:56.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:56.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:56.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:56.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:56.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:56.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:56.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:11:56.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: osd.7 [v2:172.21.15.161:6824/319989768,v1:172.21.15.161:6825/319989768] boot 2024-06-01T02:11:56.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: osdmap e189: 8 total, 8 up, 8 in 2024-06-01T02:11:56.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T02:11:56.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:56.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:56.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:56.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:56.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:56.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:56.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:56.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:56.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:56.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:56.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:56.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:56.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:56.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:56.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:56.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:56.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:56.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:56.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:56.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:56.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:56.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:56.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:56.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:56.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:56.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:56.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:56.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:56.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:56.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:56.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:56.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:56.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:56.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:56.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:56.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:56.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:56.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:56.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:56.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:56.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:56.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:56.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:56.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:56.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:56.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:56.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:56.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:56.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:56.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:56.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:56.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:56.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:56.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:56.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:56.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:56.797 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:11:56.544+0000 7fcedaabb700 -1 mgr.server reply reply (16) Device or resource busy unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-06-01T02:11:57.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:11:56] "GET /metrics HTTP/1.1" 200 38618 "" "Prometheus/2.33.4" 2024-06-01T02:11:57.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:57.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:57.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: pgmap v305: 161 pgs: 24 active+undersized+degraded, 38 active+undersized, 99 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 91/726 objects degraded (12.534%) 2024-06-01T02:11:57.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:57.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:57.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:57.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:57.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:57.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:57.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:57.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:57.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:57.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:57.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:57.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: osdmap e190: 8 total, 8 up, 8 in 2024-06-01T02:11:57.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:57.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:57.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:57.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:57.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:57.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:57.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:57.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:57.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:57.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:57.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: Health check update: Degraded data redundancy: 91/726 objects degraded (12.534%), 24 pgs degraded (PG_DEGRADED) 2024-06-01T02:11:57.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:57.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:57.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:57.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:57.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:57.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:57.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:57.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:57.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:57.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:57.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:57.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:57.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-06-01T02:11:57.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:57.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:57.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: pgmap v305: 161 pgs: 24 active+undersized+degraded, 38 active+undersized, 99 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 91/726 objects degraded (12.534%) 2024-06-01T02:11:57.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:57.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:57.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:57.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:57.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:57.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:57.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:57.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:57.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:57.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:57.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:57.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: osdmap e190: 8 total, 8 up, 8 in 2024-06-01T02:11:57.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:57.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:57.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:57.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:57.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:57.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:57.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:57.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:57.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:57.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:57.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: Health check update: Degraded data redundancy: 91/726 objects degraded (12.534%), 24 pgs degraded (PG_DEGRADED) 2024-06-01T02:11:57.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:57.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:57.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:57.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:57.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:57.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: pgmap v305: 161 pgs: 24 active+undersized+degraded, 38 active+undersized, 99 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 91/726 objects degraded (12.534%) 2024-06-01T02:11:57.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:57.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:57.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:57.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:57.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:57.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:57.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:57.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:57.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:57.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:57.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:57.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: osdmap e190: 8 total, 8 up, 8 in 2024-06-01T02:11:57.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:57.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:57.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:57.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:57.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:57.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:57.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:57.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:57.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:57.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:57.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: Health check update: Degraded data redundancy: 91/726 objects degraded (12.534%), 24 pgs degraded (PG_DEGRADED) 2024-06-01T02:11:57.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:57.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:57.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:11:57.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:57.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:57.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:57.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:57.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:57.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:57.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:57.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:57.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:57.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-06-01T02:11:57.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:11:57.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:57.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:57.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:57.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:11:57.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:11:57.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:57.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:57.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:11:57.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-06-01T02:11:58.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:57 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:58.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:57 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:58.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:57 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:58.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:57 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:58.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:57 smithi116 ceph-mon[89502]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-06-01T02:11:58.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:57 smithi116 ceph-mon[89502]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-06-01T02:11:58.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:57 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:58.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:57 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:58.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:57 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:58.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:57 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:58.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:57 smithi116 ceph-mon[100606]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-06-01T02:11:58.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:57 smithi116 ceph-mon[100606]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-06-01T02:11:58.396 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:57 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:58.396 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:57 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:58.396 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:57 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:58.396 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:57 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:11:58.396 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:57 smithi161 ceph-mon[84406]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-06-01T02:11:58.396 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:57 smithi161 ceph-mon[84406]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-06-01T02:11:59.193 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:58 smithi161 ceph-mon[84406]: pgmap v307: 161 pgs: 12 peering, 20 active+undersized+degraded, 26 active+undersized, 103 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 82/726 objects degraded (11.295%) 2024-06-01T02:11:59.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:58 smithi116 ceph-mon[89502]: pgmap v307: 161 pgs: 12 peering, 20 active+undersized+degraded, 26 active+undersized, 103 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 82/726 objects degraded (11.295%) 2024-06-01T02:11:59.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:58 smithi116 ceph-mon[100606]: pgmap v307: 161 pgs: 12 peering, 20 active+undersized+degraded, 26 active+undersized, 103 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 82/726 objects degraded (11.295%) 2024-06-01T02:12:00.042 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:11:59 smithi116 ceph-mon[100606]: pgmap v308: 161 pgs: 12 peering, 149 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:12:00.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:11:59 smithi116 ceph-mon[89502]: pgmap v308: 161 pgs: 12 peering, 149 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:12:00.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:11:59 smithi161 ceph-mon[84406]: pgmap v308: 161 pgs: 12 peering, 149 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:12:01.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:00 smithi116 ceph-mon[100606]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 82/726 objects degraded (11.295%), 20 pgs degraded) 2024-06-01T02:12:01.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:00 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:01.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:00 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:12:01.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:00 smithi116 ceph-mon[89502]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 82/726 objects degraded (11.295%), 20 pgs degraded) 2024-06-01T02:12:01.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:01.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:12:01.307 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:00 smithi161 ceph-mon[84406]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 82/726 objects degraded (11.295%), 20 pgs degraded) 2024-06-01T02:12:01.307 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:00 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:01.307 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:00 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:12:03.042 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:02 smithi116 ceph-mon[100606]: pgmap v309: 161 pgs: 161 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 718 B/s rd, 0 op/s 2024-06-01T02:12:03.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:02 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:03.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:02 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:03.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:02 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:03.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:02 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:03.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:02 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:03.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:02 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:03.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:02 smithi116 ceph-mon[89502]: pgmap v309: 161 pgs: 161 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 718 B/s rd, 0 op/s 2024-06-01T02:12:03.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:03.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:03.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:03.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:03.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:03.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:02 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:03.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:02 smithi161 ceph-mon[84406]: pgmap v309: 161 pgs: 161 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 718 B/s rd, 0 op/s 2024-06-01T02:12:03.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:02 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:03.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:02 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:03.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:02 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:03.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:02 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:03.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:02 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:03.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:02 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:04.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:03 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:12:04.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:03 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:12:04.193 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:03 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:12:05.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:04 smithi116 ceph-mon[89502]: pgmap v310: 161 pgs: 161 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:12:05.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:04 smithi116 ceph-mon[100606]: pgmap v310: 161 pgs: 161 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:12:05.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:04 smithi161 ceph-mon[84406]: pgmap v310: 161 pgs: 161 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:12:07.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:06 smithi116 ceph-mon[89502]: pgmap v311: 161 pgs: 161 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:12:07.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:06 smithi116 ceph-mon[100606]: pgmap v311: 161 pgs: 161 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:12:07.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:06 smithi161 ceph-mon[84406]: pgmap v311: 161 pgs: 161 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:12:07.195 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:12:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:12:06] "GET /metrics HTTP/1.1" 200 38626 "" "Prometheus/2.33.4" 2024-06-01T02:12:09.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:08 smithi161 ceph-mon[84406]: pgmap v312: 161 pgs: 161 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 920 B/s rd, 0 op/s 2024-06-01T02:12:09.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:08 smithi116 ceph-mon[89502]: pgmap v312: 161 pgs: 161 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 920 B/s rd, 0 op/s 2024-06-01T02:12:09.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:08 smithi116 ceph-mon[100606]: pgmap v312: 161 pgs: 161 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 920 B/s rd, 0 op/s 2024-06-01T02:12:11.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:10 smithi161 ceph-mon[84406]: pgmap v313: 161 pgs: 161 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:12:11.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:10 smithi116 ceph-mon[89502]: pgmap v313: 161 pgs: 161 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:12:11.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:10 smithi116 ceph-mon[100606]: pgmap v313: 161 pgs: 161 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:12:12.281 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:12 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:12.281 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:12 smithi161 ceph-mon[84406]: pgmap v314: 161 pgs: 161 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:12:12.281 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:12 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-06-01T02:12:12.281 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:12 smithi161 ceph-mon[84406]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-06-01T02:12:12.281 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:12 smithi161 ceph-mon[84406]: Upgrade: osd.7 is safe to restart 2024-06-01T02:12:12.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:12 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:12.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:12 smithi116 ceph-mon[89502]: pgmap v314: 161 pgs: 161 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:12:12.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:12 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-06-01T02:12:12.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:12 smithi116 ceph-mon[89502]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-06-01T02:12:12.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:12 smithi116 ceph-mon[89502]: Upgrade: osd.7 is safe to restart 2024-06-01T02:12:12.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:12 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:12.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:12 smithi116 ceph-mon[100606]: pgmap v314: 161 pgs: 161 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:12:12.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:12 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-06-01T02:12:12.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:12 smithi116 ceph-mon[100606]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-06-01T02:12:12.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:12 smithi116 ceph-mon[100606]: Upgrade: osd.7 is safe to restart 2024-06-01T02:12:13.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:13 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:12:13.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:13 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:12:13.335 INFO:teuthology.orchestra.run.smithi116.stdout:true 2024-06-01T02:12:13.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:13 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:12:13.995 INFO:teuthology.orchestra.run.smithi116.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-01T02:12:13.995 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi116 smithi116 running 0s ago 20m - - 2024-06-01T02:12:13.995 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi161 smithi161 running 11s ago 20m - - 2024-06-01T02:12:13.995 INFO:teuthology.orchestra.run.smithi116.stdout:alertmanager.a smithi116 *:9093,9094 running (10m) 0s ago 15m 22.3M - 0.23.0 ba2b418f427c e0c297162ded 2024-06-01T02:12:13.995 INFO:teuthology.orchestra.run.smithi116.stdout:grafana.a smithi161 *:3000 running (9m) 11s ago 15m 52.5M - 8.3.5 dad864ee21e9 00b0a78ee88e 2024-06-01T02:12:13.995 INFO:teuthology.orchestra.run.smithi116.stdout:iscsi.foo.smithi116.gjuxzb smithi116 running (8m) 0s ago 13m 62.9M - 3.5 e1d6a67b021e 0e191fe3e3bb 2024-06-01T02:12:13.995 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.x smithi161 *:8443,9283,8765 running (9m) 11s ago 20m 538M - 19.0.0-2946-g3d89accf b045ac545f82 2d7fbb85e5f7 2024-06-01T02:12:13.995 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.y smithi116 *:8443,9283,8765 running (8m) 0s ago 21m 456M - 19.0.0-2946-g3d89accf b045ac545f82 bc9627c27a57 2024-06-01T02:12:13.996 INFO:teuthology.orchestra.run.smithi116.stdout:mon.a smithi116 running (7m) 0s ago 21m 75.6M 2048M 19.0.0-2946-g3d89accf b045ac545f82 b5e9b6f873ff 2024-06-01T02:12:13.996 INFO:teuthology.orchestra.run.smithi116.stdout:mon.b smithi161 running (6m) 11s ago 20m 51.7M 2048M 19.0.0-2946-g3d89accf b045ac545f82 aaa08c03cfc8 2024-06-01T02:12:13.996 INFO:teuthology.orchestra.run.smithi116.stdout:mon.c smithi116 running (6m) 0s ago 20m 58.6M 2048M 19.0.0-2946-g3d89accf b045ac545f82 bd0fb0e603fa 2024-06-01T02:12:13.996 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.a smithi116 *:9100 running (10m) 0s ago 15m 17.7M - 1.3.1 1dbe0e931976 e347748d17bb 2024-06-01T02:12:13.996 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.b smithi161 *:9100 running (9m) 11s ago 15m 19.8M - 1.3.1 1dbe0e931976 6e66182f5a1a 2024-06-01T02:12:13.996 INFO:teuthology.orchestra.run.smithi116.stdout:osd.0 smithi116 running (5m) 0s ago 19m 55.5M 4096M 19.0.0-2946-g3d89accf b045ac545f82 b4313d121b9d 2024-06-01T02:12:13.996 INFO:teuthology.orchestra.run.smithi116.stdout:osd.1 smithi116 running (4m) 0s ago 19m 57.2M 4096M 19.0.0-2946-g3d89accf b045ac545f82 dccdbeac5aae 2024-06-01T02:12:13.996 INFO:teuthology.orchestra.run.smithi116.stdout:osd.2 smithi116 running (4m) 0s ago 18m 51.3M 4096M 19.0.0-2946-g3d89accf b045ac545f82 1ad95efda404 2024-06-01T02:12:13.996 INFO:teuthology.orchestra.run.smithi116.stdout:osd.3 smithi116 running (3m) 0s ago 18m 54.4M 4096M 19.0.0-2946-g3d89accf b045ac545f82 bd00ad2c2556 2024-06-01T02:12:13.996 INFO:teuthology.orchestra.run.smithi116.stdout:osd.4 smithi161 running (2m) 11s ago 18m 52.7M 4096M 19.0.0-2946-g3d89accf b045ac545f82 0084b9edf7a0 2024-06-01T02:12:13.996 INFO:teuthology.orchestra.run.smithi116.stdout:osd.5 smithi161 running (92s) 11s ago 17m 49.0M 4096M 19.0.0-2946-g3d89accf b045ac545f82 b3849cdeb707 2024-06-01T02:12:13.996 INFO:teuthology.orchestra.run.smithi116.stdout:osd.6 smithi161 running (53s) 11s ago 17m 45.2M 4096M 19.0.0-2946-g3d89accf b045ac545f82 6b87d9299c8d 2024-06-01T02:12:13.996 INFO:teuthology.orchestra.run.smithi116.stdout:osd.7 smithi161 running (24s) 11s ago 16m 44.9M 4096M 19.0.0-2946-g3d89accf b045ac545f82 c98098e14cc9 2024-06-01T02:12:13.997 INFO:teuthology.orchestra.run.smithi116.stdout:prometheus.a smithi161 *:9095 running (8m) 11s ago 16m 79.0M - 2.33.4 514e6a882f6e e977f6ca1d73 2024-06-01T02:12:13.997 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi116.vzjrxj smithi116 *:8000 running (14m) 0s ago 14m 78.2M - 17.2.0 e1d6a67b021e df4fe3c1dbc3 2024-06-01T02:12:13.997 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi161.ashqxz smithi161 *:8000 running (14m) 11s ago 14m 78.6M - 17.2.0 e1d6a67b021e 3eecc1f3e2d3 2024-06-01T02:12:13.997 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi116.hokbmf smithi116 *:80 running (14m) 0s ago 14m 78.6M - 17.2.0 e1d6a67b021e 10950ed75dbd 2024-06-01T02:12:13.997 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi161.axjlzw smithi161 *:80 running (14m) 11s ago 14m 77.8M - 17.2.0 e1d6a67b021e 0b5d5ce47772 2024-06-01T02:12:14.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:14 smithi116 ceph-mon[100606]: Upgrade: Updating osd.7 2024-06-01T02:12:14.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-01T02:12:14.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:14.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:14 smithi116 ceph-mon[100606]: Deploying daemon osd.7 on smithi161 2024-06-01T02:12:14.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:14 smithi116 ceph-mon[100606]: pgmap v315: 161 pgs: 161 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:12:14.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:14 smithi116 ceph-mon[100606]: from='client.44631 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:12:14.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:14 smithi116 ceph-mon[100606]: from='client.44637 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:12:14.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:14.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:14.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:14 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:14.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:14 smithi116 ceph-mon[100606]: from='client.44643 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:12:14.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:14 smithi116 ceph-mon[89502]: Upgrade: Updating osd.7 2024-06-01T02:12:14.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-01T02:12:14.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:14.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:14 smithi116 ceph-mon[89502]: Deploying daemon osd.7 on smithi161 2024-06-01T02:12:14.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:14 smithi116 ceph-mon[89502]: pgmap v315: 161 pgs: 161 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:12:14.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:14 smithi116 ceph-mon[89502]: from='client.44631 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:12:14.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:14 smithi116 ceph-mon[89502]: from='client.44637 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:12:14.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:14.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:14.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:14 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:14.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:14 smithi116 ceph-mon[89502]: from='client.44643 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:12:14.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:14 smithi161 ceph-mon[84406]: Upgrade: Updating osd.7 2024-06-01T02:12:14.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-01T02:12:14.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:14.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:14 smithi161 ceph-mon[84406]: Deploying daemon osd.7 on smithi161 2024-06-01T02:12:14.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:14 smithi161 ceph-mon[84406]: pgmap v315: 161 pgs: 161 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:12:14.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:14 smithi161 ceph-mon[84406]: from='client.44631 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:12:14.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:14 smithi161 ceph-mon[84406]: from='client.44637 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:12:14.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:14.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:14.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:14 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:14.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:14 smithi161 ceph-mon[84406]: from='client.44643 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:12:14.387 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:12:14.388 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": { 2024-06-01T02:12:14.388 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 3 2024-06-01T02:12:14.388 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:12:14.388 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": { 2024-06-01T02:12:14.388 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-01T02:12:14.388 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:12:14.388 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": { 2024-06-01T02:12:14.388 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 8 2024-06-01T02:12:14.388 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:12:14.388 INFO:teuthology.orchestra.run.smithi116.stdout: "rgw": { 2024-06-01T02:12:14.388 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-06-01T02:12:14.388 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:12:14.389 INFO:teuthology.orchestra.run.smithi116.stdout: "overall": { 2024-06-01T02:12:14.389 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4, 2024-06-01T02:12:14.389 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 13 2024-06-01T02:12:14.389 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T02:12:14.389 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:12:14.723 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:12:14.723 INFO:teuthology.orchestra.run.smithi116.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-01T02:12:14.723 INFO:teuthology.orchestra.run.smithi116.stdout: "in_progress": true, 2024-06-01T02:12:14.723 INFO:teuthology.orchestra.run.smithi116.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-01T02:12:14.723 INFO:teuthology.orchestra.run.smithi116.stdout: "services_complete": [ 2024-06-01T02:12:14.723 INFO:teuthology.orchestra.run.smithi116.stdout: "mon", 2024-06-01T02:12:14.723 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr", 2024-06-01T02:12:14.723 INFO:teuthology.orchestra.run.smithi116.stdout: "osd" 2024-06-01T02:12:14.724 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-06-01T02:12:14.724 INFO:teuthology.orchestra.run.smithi116.stdout: "progress": "13/23 daemons upgraded", 2024-06-01T02:12:14.724 INFO:teuthology.orchestra.run.smithi116.stdout: "message": "Currently upgrading osd daemons", 2024-06-01T02:12:14.724 INFO:teuthology.orchestra.run.smithi116.stdout: "is_paused": false 2024-06-01T02:12:14.724 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:12:15.116 INFO:teuthology.orchestra.run.smithi116.stdout:HEALTH_WARN all OSDs are running squid or later but require_osd_release < squid 2024-06-01T02:12:15.116 INFO:teuthology.orchestra.run.smithi116.stdout:[WRN] OSD_UPGRADE_FINISHED: all OSDs are running squid or later but require_osd_release < squid 2024-06-01T02:12:15.116 INFO:teuthology.orchestra.run.smithi116.stdout: all OSDs are running squid or later but require_osd_release < squid 2024-06-01T02:12:15.283 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:15 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/209232783' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:15.283 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:15 smithi161 ceph-mon[84406]: from='client.64615 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:12:15.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:15 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/209232783' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:15.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:15 smithi116 ceph-mon[100606]: from='client.64615 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:12:15.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:15 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/209232783' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:15.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:15 smithi116 ceph-mon[89502]: from='client.64615 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:12:16.308 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:16 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/231880859' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:12:16.308 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:16 smithi161 ceph-mon[84406]: pgmap v316: 161 pgs: 161 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:12:16.309 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:12:16.542 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:16 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/231880859' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:12:16.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:16 smithi116 ceph-mon[100606]: pgmap v316: 161 pgs: 161 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:12:16.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:12:16.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:16 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/231880859' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:12:16.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:16 smithi116 ceph-mon[89502]: pgmap v316: 161 pgs: 161 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:12:16.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:12:16.645 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:16 smithi161 systemd[1]: Stopping Ceph osd.7 for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:12:16.944 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:12:16 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:12:16] "GET /metrics HTTP/1.1" 200 38629 "" "Prometheus/2.33.4" 2024-06-01T02:12:16.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:16 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[126755]: 2024-06-01T02:12:16.728+0000 7f3bf6507700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-01T02:12:16.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:16 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[126755]: 2024-06-01T02:12:16.728+0000 7f3bf6507700 -1 osd.7 190 *** Got signal Terminated *** 2024-06-01T02:12:16.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:16 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[126755]: 2024-06-01T02:12:16.728+0000 7f3bf6507700 -1 osd.7 190 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-01T02:12:17.398 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:17 smithi161 ceph-mon[84406]: osd.7 marked itself down and dead 2024-06-01T02:12:17.542 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:17 smithi116 ceph-mon[89502]: osd.7 marked itself down and dead 2024-06-01T02:12:17.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:17 smithi116 ceph-mon[100606]: osd.7 marked itself down and dead 2024-06-01T02:12:17.694 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:17 smithi161 podman[130283]: 2024-06-01 02:12:17.403638136 +0000 UTC m=+0.820241848 container died c98098e14cc9373f55402d74ae0d4184930f1acf5db306131fb434c838c87ad3 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-06-01T02:12:18.049 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:17 smithi161 podman[130283]: 2024-06-01 02:12:17.795922122 +0000 UTC m=+1.212525837 container cleanup c98098e14cc9373f55402d74ae0d4184930f1acf5db306131fb434c838c87ad3 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-06-01T02:12:18.049 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:17 smithi161 bash[130283]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7 2024-06-01T02:12:18.049 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:17 smithi161 podman[130300]: 2024-06-01 02:12:17.964092237 +0000 UTC m=+0.560425959 container remove c98098e14cc9373f55402d74ae0d4184930f1acf5db306131fb434c838c87ad3 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-06-01T02:12:18.354 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:18 smithi161 ceph-mon[84406]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:12:18.354 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:18 smithi161 ceph-mon[84406]: osdmap e191: 8 total, 7 up, 8 in 2024-06-01T02:12:18.354 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:18 smithi161 ceph-mon[84406]: pgmap v318: 161 pgs: 20 stale+active+clean, 141 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:12:18.444 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:18 smithi116 ceph-mon[100606]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:12:18.444 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:18 smithi116 ceph-mon[100606]: osdmap e191: 8 total, 7 up, 8 in 2024-06-01T02:12:18.444 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:18 smithi116 ceph-mon[100606]: pgmap v318: 161 pgs: 20 stale+active+clean, 141 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:12:18.445 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:18 smithi116 ceph-mon[89502]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-01T02:12:18.445 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:18 smithi116 ceph-mon[89502]: osdmap e191: 8 total, 7 up, 8 in 2024-06-01T02:12:18.445 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:18 smithi116 ceph-mon[89502]: pgmap v318: 161 pgs: 20 stale+active+clean, 141 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-01T02:12:19.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:19 smithi116 ceph-mon[89502]: osdmap e192: 8 total, 7 up, 8 in 2024-06-01T02:12:19.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:19 smithi116 ceph-mon[100606]: osdmap e192: 8 total, 7 up, 8 in 2024-06-01T02:12:19.324 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:19 smithi161 ceph-mon[84406]: osdmap e192: 8 total, 7 up, 8 in 2024-06-01T02:12:19.324 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:19 smithi161 podman[130458]: 2024-06-01 02:12:19.214214431 +0000 UTC m=+0.018323438 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:12:19.603 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:19 smithi161 podman[130458]: 2024-06-01 02:12:19.329706566 +0000 UTC m=+0.133815546 container create cc64c125b349928c5316ba2770e0e64a68c742b8d96ae55bdf055556eb7d95b1 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-06-01T02:12:19.603 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:19 smithi161 podman[130458]: 2024-06-01 02:12:19.54087919 +0000 UTC m=+0.344988164 container init cc64c125b349928c5316ba2770e0e64a68c742b8d96ae55bdf055556eb7d95b1 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-06-01T02:12:19.603 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:19 smithi161 podman[130458]: 2024-06-01 02:12:19.544155975 +0000 UTC m=+0.348264953 container start cc64c125b349928c5316ba2770e0e64a68c742b8d96ae55bdf055556eb7d95b1 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-06-01T02:12:19.867 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:19 smithi161 podman[130458]: 2024-06-01 02:12:19.607362164 +0000 UTC m=+0.411471139 container attach cc64c125b349928c5316ba2770e0e64a68c742b8d96ae55bdf055556eb7d95b1 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-01T02:12:20.180 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:19 smithi161 podman[130458]: 2024-06-01 02:12:19.872920114 +0000 UTC m=+0.677029098 container died cc64c125b349928c5316ba2770e0e64a68c742b8d96ae55bdf055556eb7d95b1 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-06-01T02:12:20.180 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:20 smithi161 ceph-mon[84406]: pgmap v320: 161 pgs: 26 peering, 20 stale+active+clean, 115 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:12:20.542 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:20 smithi116 ceph-mon[100606]: pgmap v320: 161 pgs: 26 peering, 20 stale+active+clean, 115 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:12:20.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:20 smithi116 ceph-mon[89502]: pgmap v320: 161 pgs: 26 peering, 20 stale+active+clean, 115 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:12:20.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:20 smithi161 podman[130585]: 2024-06-01 02:12:20.511514574 +0000 UTC m=+0.632347601 container remove cc64c125b349928c5316ba2770e0e64a68c742b8d96ae55bdf055556eb7d95b1 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-06-01T02:12:20.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:20 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.7.service: Deactivated successfully. 2024-06-01T02:12:20.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:20 smithi161 systemd[1]: Stopped Ceph osd.7 for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:12:20.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:20 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.7.service: Consumed 1.973s CPU time. 2024-06-01T02:12:20.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:20 smithi161 systemd[1]: Starting Ceph osd.7 for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:12:21.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:21 smithi161 ceph-mon[84406]: Health check failed: Degraded data redundancy: 18/726 objects degraded (2.479%), 6 pgs degraded (PG_DEGRADED) 2024-06-01T02:12:21.609 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:21 smithi161 podman[130759]: 2024-06-01 02:12:21.410359524 +0000 UTC m=+0.023238765 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:12:21.609 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:21 smithi161 podman[130759]: 2024-06-01 02:12:21.532881536 +0000 UTC m=+0.145760752 container create 3b613cbeeb5594df651be9123114843dec33f326aa5d16d092e5541c4b949073 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-activate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-06-01T02:12:21.792 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:21 smithi116 ceph-mon[89502]: Health check failed: Degraded data redundancy: 18/726 objects degraded (2.479%), 6 pgs degraded (PG_DEGRADED) 2024-06-01T02:12:21.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:21 smithi116 ceph-mon[100606]: Health check failed: Degraded data redundancy: 18/726 objects degraded (2.479%), 6 pgs degraded (PG_DEGRADED) 2024-06-01T02:12:21.943 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:21 smithi161 podman[130759]: 2024-06-01 02:12:21.718986088 +0000 UTC m=+0.331865320 container init 3b613cbeeb5594df651be9123114843dec33f326aa5d16d092e5541c4b949073 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-activate, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, ceph=True) 2024-06-01T02:12:21.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:21 smithi161 podman[130759]: 2024-06-01 02:12:21.722633773 +0000 UTC m=+0.335512999 container start 3b613cbeeb5594df651be9123114843dec33f326aa5d16d092e5541c4b949073 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-activate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc) 2024-06-01T02:12:21.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:21 smithi161 podman[130759]: 2024-06-01 02:12:21.79357737 +0000 UTC m=+0.406456593 container attach 3b613cbeeb5594df651be9123114843dec33f326aa5d16d092e5541c4b949073 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-activate, RELEASE=squid-3d89acc, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-06-01T02:12:22.432 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:22 smithi161 ceph-mon[84406]: pgmap v321: 161 pgs: 14 active+undersized, 6 active+undersized+degraded, 26 peering, 12 stale+active+clean, 103 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 18/726 objects degraded (2.479%) 2024-06-01T02:12:22.432 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:22.433 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:22.433 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:22 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:22.433 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:22 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-activate[130798]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-06-01T02:12:22.433 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:22 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-activate[130798]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-7 --no-mon-config --dev /dev/mapper/vg_nvme-lv_1 2024-06-01T02:12:22.433 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:22 smithi161 bash[130759]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-06-01T02:12:22.433 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:22 smithi161 bash[130759]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-7 --no-mon-config --dev /dev/mapper/vg_nvme-lv_1 2024-06-01T02:12:22.433 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:22 smithi161 bash[130759]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_1 2024-06-01T02:12:22.433 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:22 smithi161 bash[130759]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0 2024-06-01T02:12:22.434 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:22 smithi161 bash[130759]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-7/block 2024-06-01T02:12:22.434 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:22 smithi161 bash[130759]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-06-01T02:12:22.434 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:22 smithi161 bash[130759]: --> ceph-volume raw activate successful for osd ID: 7 2024-06-01T02:12:22.434 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:22 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-activate[130798]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_1 2024-06-01T02:12:22.434 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:22 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-activate[130798]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0 2024-06-01T02:12:22.434 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:22 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-activate[130798]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-7/block 2024-06-01T02:12:22.434 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:22 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-activate[130798]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-06-01T02:12:22.434 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:22 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-activate[130798]: --> ceph-volume raw activate successful for osd ID: 7 2024-06-01T02:12:22.542 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:22 smithi116 ceph-mon[89502]: pgmap v321: 161 pgs: 14 active+undersized, 6 active+undersized+degraded, 26 peering, 12 stale+active+clean, 103 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 18/726 objects degraded (2.479%) 2024-06-01T02:12:22.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:22.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:22.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:22 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:22.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:22 smithi116 ceph-mon[100606]: pgmap v321: 161 pgs: 14 active+undersized, 6 active+undersized+degraded, 26 peering, 12 stale+active+clean, 103 active+clean; 585 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 18/726 objects degraded (2.479%) 2024-06-01T02:12:22.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:22.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:22.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:22 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:22.694 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:22 smithi161 podman[130759]: 2024-06-01 02:12:22.567190596 +0000 UTC m=+1.180069866 container died 3b613cbeeb5594df651be9123114843dec33f326aa5d16d092e5541c4b949073 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-activate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD) 2024-06-01T02:12:23.112 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:23 smithi161 podman[131057]: 2024-06-01 02:12:23.090900548 +0000 UTC m=+0.847452952 container remove 3b613cbeeb5594df651be9123114843dec33f326aa5d16d092e5541c4b949073 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-activate, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-06-01T02:12:23.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:23 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:12:23.496 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:23 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:12:23.497 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:23 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:12:24.415 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:24 smithi161 ceph-mon[84406]: pgmap v322: 161 pgs: 38 active+undersized, 24 active+undersized+degraded, 99 active+clean; 585 KiB data, 251 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 91/726 objects degraded (12.534%) 2024-06-01T02:12:24.416 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:24 smithi161 podman[131243]: 2024-06-01 02:12:24.179190368 +0000 UTC m=+0.021793961 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:12:24.416 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:24 smithi161 podman[131243]: 2024-06-01 02:12:24.305865898 +0000 UTC m=+0.148469464 container create 400bb59c4249e6665af48d3d48b81ca9cc5a23e28c532b1b61a262fb30d3e7f4 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-06-01T02:12:24.716 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:24 smithi116 ceph-mon[89502]: pgmap v322: 161 pgs: 38 active+undersized, 24 active+undersized+degraded, 99 active+clean; 585 KiB data, 251 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 91/726 objects degraded (12.534%) 2024-06-01T02:12:24.717 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:24 smithi116 ceph-mon[100606]: pgmap v322: 161 pgs: 38 active+undersized, 24 active+undersized+degraded, 99 active+clean; 585 KiB data, 251 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 91/726 objects degraded (12.534%) 2024-06-01T02:12:24.922 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:24 smithi161 podman[131243]: 2024-06-01 02:12:24.726525621 +0000 UTC m=+0.569129204 container init 400bb59c4249e6665af48d3d48b81ca9cc5a23e28c532b1b61a262fb30d3e7f4 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-01T02:12:24.922 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:24 smithi161 podman[131243]: 2024-06-01 02:12:24.729931803 +0000 UTC m=+0.572535381 container start 400bb59c4249e6665af48d3d48b81ca9cc5a23e28c532b1b61a262fb30d3e7f4 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux ) 2024-06-01T02:12:24.922 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:24 smithi161 ceph-osd[131302]: -- 172.21.15.161:0/1077373628 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557f44349680 con 0x557f44312c00 2024-06-01T02:12:25.193 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:24 smithi161 bash[131243]: 400bb59c4249e6665af48d3d48b81ca9cc5a23e28c532b1b61a262fb30d3e7f4 2024-06-01T02:12:25.194 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:24 smithi161 systemd[1]: Started Ceph osd.7 for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:12:25.522 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:12:25.273+0000 7ff8b63ed7c0 -1 Falling back to public interface 2024-06-01T02:12:26.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:26.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:26.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:26.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:26.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:26.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:26.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:26.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:26.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:26.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:26.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:26.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:26.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:26.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:26.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:26.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:26.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:26.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:26.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:26.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:26.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:26.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:26.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:26.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:26.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:26.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:26.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:26.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:26.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:26.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:26.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:26.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:26.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:26.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:26.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:26.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:26.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:26.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:26.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:26.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:26.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:26.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:26.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:26.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:26.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:26.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:26.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:26.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:26.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:26.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:26.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:26.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:26.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:26.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:26.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:26.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:26.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:26.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:26.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:26.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:26.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:26.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:26.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:26.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:26.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:26.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:26.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:26.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:26.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:26.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:26.327 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:26.327 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:26.327 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:26.327 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:26.327 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:26.327 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:26.327 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:26.327 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:26.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:26.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:26.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:26.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:26.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:26.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:26.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:26.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:26.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:26.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:26.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:26.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:26.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:26.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:26.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:26.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:26.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:26.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:26.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:26.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:26.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:26.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:26.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:26.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:26.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:26.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:26.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.187 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:12:26] "GET /metrics HTTP/1.1" 200 38629 "" "Prometheus/2.33.4" 2024-06-01T02:12:27.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:27.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:27.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: pgmap v323: 161 pgs: 38 active+undersized, 24 active+undersized+degraded, 99 active+clean; 585 KiB data, 251 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 91/726 objects degraded (12.534%) 2024-06-01T02:12:27.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:27.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:27.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:27.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:27.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:27.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:27.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:27.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:27.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:27.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:27.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:27.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:27.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:27.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:27.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: Health check update: Degraded data redundancy: 91/726 objects degraded (12.534%), 24 pgs degraded (PG_DEGRADED) 2024-06-01T02:12:27.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:27.190 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.190 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:27.190 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:27.190 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.190 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:27.190 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.190 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:27.190 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:27.191 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.191 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:27.191 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.191 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:27.191 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:27.191 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.191 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:27.191 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.191 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:27.192 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:27.192 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.192 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.192 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.192 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.192 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:27.192 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.192 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.192 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.192 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.192 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.192 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-06-01T02:12:27.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:27.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:27.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: pgmap v323: 161 pgs: 38 active+undersized, 24 active+undersized+degraded, 99 active+clean; 585 KiB data, 251 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 91/726 objects degraded (12.534%) 2024-06-01T02:12:27.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:27.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:27.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:27.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:27.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:27.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:27.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:27.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:27.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:27.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:27.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:27.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:27.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:27.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:27.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: Health check update: Degraded data redundancy: 91/726 objects degraded (12.534%), 24 pgs degraded (PG_DEGRADED) 2024-06-01T02:12:27.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:27.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:27.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:27.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:27.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:27.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: pgmap v323: 161 pgs: 38 active+undersized, 24 active+undersized+degraded, 99 active+clean; 585 KiB data, 251 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 91/726 objects degraded (12.534%) 2024-06-01T02:12:27.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:27.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:27.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:27.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:27.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:27.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:27.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:27.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:27.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:27.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:27.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:27.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:27.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:27.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:27.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: Health check update: Degraded data redundancy: 91/726 objects degraded (12.534%), 24 pgs degraded (PG_DEGRADED) 2024-06-01T02:12:27.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:27.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:27.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:27.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:27.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:27.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:27.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:27.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:27.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:27.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:27.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:27.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:27.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:27.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-06-01T02:12:27.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:27.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:27.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:27.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:27.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:27.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:27.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:27.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:27.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:27.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:27.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:27.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:27.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-06-01T02:12:28.194 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:28 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:12:28.027+0000 7ff8b63ed7c0 -1 osd.7 190 log_to_monitors true 2024-06-01T02:12:28.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:27 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:28.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:27 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:28.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:27 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:28.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:27 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:28.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:27 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:28.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:27 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:28.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:27 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:28.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:27 smithi161 ceph-mon[84406]: pgmap v324: 161 pgs: 38 active+undersized, 24 active+undersized+degraded, 99 active+clean; 585 KiB data, 251 MiB used, 715 GiB / 715 GiB avail; 950 B/s rd, 0 op/s; 91/726 objects degraded (12.534%) 2024-06-01T02:12:28.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:27 smithi161 ceph-mon[84406]: Upgrade: Setting container_image for all osd 2024-06-01T02:12:28.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-06-01T02:12:28.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-06-01T02:12:28.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-06-01T02:12:28.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-06-01T02:12:28.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-06-01T02:12:28.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-06-01T02:12:28.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-06-01T02:12:28.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-06-01T02:12:28.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-06-01T02:12:28.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-06-01T02:12:28.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-06-01T02:12:28.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.6"}]: dispatch 2024-06-01T02:12:28.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.6"}]': finished 2024-06-01T02:12:28.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.7"}]: dispatch 2024-06-01T02:12:28.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.7"}]': finished 2024-06-01T02:12:28.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-06-01T02:12:28.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:27 smithi161 ceph-mon[84406]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-06-01T02:12:28.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:28.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:28.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:28.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:28.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:28.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:28.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:28.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[89502]: pgmap v324: 161 pgs: 38 active+undersized, 24 active+undersized+degraded, 99 active+clean; 585 KiB data, 251 MiB used, 715 GiB / 715 GiB avail; 950 B/s rd, 0 op/s; 91/726 objects degraded (12.534%) 2024-06-01T02:12:28.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[89502]: Upgrade: Setting container_image for all osd 2024-06-01T02:12:28.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-06-01T02:12:28.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-06-01T02:12:28.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-06-01T02:12:28.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-06-01T02:12:28.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-06-01T02:12:28.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-06-01T02:12:28.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-06-01T02:12:28.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-06-01T02:12:28.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-06-01T02:12:28.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-06-01T02:12:28.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-06-01T02:12:28.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.6"}]: dispatch 2024-06-01T02:12:28.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.6"}]': finished 2024-06-01T02:12:28.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.7"}]: dispatch 2024-06-01T02:12:28.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.7"}]': finished 2024-06-01T02:12:28.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-06-01T02:12:28.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[89502]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-06-01T02:12:28.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:28.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:28.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:28.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:28.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:28.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:28.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:28.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[100606]: pgmap v324: 161 pgs: 38 active+undersized, 24 active+undersized+degraded, 99 active+clean; 585 KiB data, 251 MiB used, 715 GiB / 715 GiB avail; 950 B/s rd, 0 op/s; 91/726 objects degraded (12.534%) 2024-06-01T02:12:28.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[100606]: Upgrade: Setting container_image for all osd 2024-06-01T02:12:28.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-06-01T02:12:28.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-06-01T02:12:28.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-06-01T02:12:28.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-06-01T02:12:28.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-06-01T02:12:28.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-06-01T02:12:28.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-06-01T02:12:28.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-06-01T02:12:28.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-06-01T02:12:28.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-06-01T02:12:28.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-06-01T02:12:28.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.6"}]: dispatch 2024-06-01T02:12:28.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.6"}]': finished 2024-06-01T02:12:28.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.7"}]: dispatch 2024-06-01T02:12:28.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.7"}]': finished 2024-06-01T02:12:28.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-06-01T02:12:28.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:27 smithi116 ceph-mon[100606]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-06-01T02:12:29.275 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:28 smithi161 ceph-mon[84406]: Upgrade: Setting require_osd_release to 19 squid 2024-06-01T02:12:29.275 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:28 smithi161 ceph-mon[84406]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-06-01T02:12:29.276 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-06-01T02:12:29.276 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:28 smithi161 ceph-mon[84406]: osdmap e193: 8 total, 7 up, 8 in 2024-06-01T02:12:29.276 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:29.276 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:29.276 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:28 smithi161 ceph-mon[84406]: from='osd.7 [v2:172.21.15.161:6824/4276809814,v1:172.21.15.161:6825/4276809814]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-01T02:12:29.276 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:28 smithi161 ceph-mon[84406]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-01T02:12:29.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:28 smithi116 ceph-mon[89502]: Upgrade: Setting require_osd_release to 19 squid 2024-06-01T02:12:29.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:28 smithi116 ceph-mon[89502]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-06-01T02:12:29.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-06-01T02:12:29.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:28 smithi116 ceph-mon[89502]: osdmap e193: 8 total, 7 up, 8 in 2024-06-01T02:12:29.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:29.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:29.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:28 smithi116 ceph-mon[89502]: from='osd.7 [v2:172.21.15.161:6824/4276809814,v1:172.21.15.161:6825/4276809814]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-01T02:12:29.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:28 smithi116 ceph-mon[89502]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-01T02:12:29.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:28 smithi116 ceph-mon[100606]: Upgrade: Setting require_osd_release to 19 squid 2024-06-01T02:12:29.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:28 smithi116 ceph-mon[100606]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-06-01T02:12:29.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-06-01T02:12:29.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:28 smithi116 ceph-mon[100606]: osdmap e193: 8 total, 7 up, 8 in 2024-06-01T02:12:29.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:29.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:29.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:28 smithi116 ceph-mon[100606]: from='osd.7 [v2:172.21.15.161:6824/4276809814,v1:172.21.15.161:6825/4276809814]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-01T02:12:29.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:28 smithi116 ceph-mon[100606]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-01T02:12:29.646 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:12:29 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:12:29.508+0000 7ff8a383c700 -1 osd.7 190 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-01T02:12:30.269 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:29 smithi116 ceph-mon[89502]: Upgrade: Setting container_image for all mds 2024-06-01T02:12:30.269 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:29 smithi116 ceph-mon[89502]: pgmap v326: 161 pgs: 38 active+undersized, 24 active+undersized+degraded, 99 active+clean; 585 KiB data, 251 MiB used, 715 GiB / 715 GiB avail; 950 B/s rd, 0 op/s; 91/726 objects degraded (12.534%) 2024-06-01T02:12:30.269 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:29 smithi116 ceph-mon[89502]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-01T02:12:30.269 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:29 smithi116 ceph-mon[89502]: osdmap e194: 8 total, 7 up, 8 in 2024-06-01T02:12:30.269 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:29 smithi116 ceph-mon[89502]: from='osd.7 [v2:172.21.15.161:6824/4276809814,v1:172.21.15.161:6825/4276809814]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:12:30.269 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:29 smithi116 ceph-mon[89502]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:12:30.269 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:29 smithi116 ceph-mon[89502]: from='osd.7 ' entity='osd.7' 2024-06-01T02:12:30.269 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:30.269 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi116.vzjrxj", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T02:12:30.269 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:30.270 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:29 smithi116 ceph-mon[100606]: Upgrade: Setting container_image for all mds 2024-06-01T02:12:30.270 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:29 smithi116 ceph-mon[100606]: pgmap v326: 161 pgs: 38 active+undersized, 24 active+undersized+degraded, 99 active+clean; 585 KiB data, 251 MiB used, 715 GiB / 715 GiB avail; 950 B/s rd, 0 op/s; 91/726 objects degraded (12.534%) 2024-06-01T02:12:30.270 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:29 smithi116 ceph-mon[100606]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-01T02:12:30.270 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:29 smithi116 ceph-mon[100606]: osdmap e194: 8 total, 7 up, 8 in 2024-06-01T02:12:30.270 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:29 smithi116 ceph-mon[100606]: from='osd.7 [v2:172.21.15.161:6824/4276809814,v1:172.21.15.161:6825/4276809814]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:12:30.270 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:29 smithi116 ceph-mon[100606]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:12:30.270 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:29 smithi116 ceph-mon[100606]: from='osd.7 ' entity='osd.7' 2024-06-01T02:12:30.270 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:30.270 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi116.vzjrxj", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T02:12:30.270 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:30.281 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:29 smithi161 ceph-mon[84406]: Upgrade: Setting container_image for all mds 2024-06-01T02:12:30.281 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:29 smithi161 ceph-mon[84406]: pgmap v326: 161 pgs: 38 active+undersized, 24 active+undersized+degraded, 99 active+clean; 585 KiB data, 251 MiB used, 715 GiB / 715 GiB avail; 950 B/s rd, 0 op/s; 91/726 objects degraded (12.534%) 2024-06-01T02:12:30.281 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:29 smithi161 ceph-mon[84406]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-01T02:12:30.281 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:29 smithi161 ceph-mon[84406]: osdmap e194: 8 total, 7 up, 8 in 2024-06-01T02:12:30.281 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:29 smithi161 ceph-mon[84406]: from='osd.7 [v2:172.21.15.161:6824/4276809814,v1:172.21.15.161:6825/4276809814]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:12:30.281 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:29 smithi161 ceph-mon[84406]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-06-01T02:12:30.281 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:29 smithi161 ceph-mon[84406]: from='osd.7 ' entity='osd.7' 2024-06-01T02:12:30.281 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:30.281 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi116.vzjrxj", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T02:12:30.281 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:31.279 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:30 smithi161 ceph-mon[84406]: Upgrade: Updating rgw.foo.smithi116.vzjrxj (1/4) 2024-06-01T02:12:31.280 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:30 smithi161 ceph-mon[84406]: Deploying daemon rgw.foo.smithi116.vzjrxj on smithi116 2024-06-01T02:12:31.280 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:30 smithi161 ceph-mon[84406]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:12:31.280 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:30 smithi161 ceph-mon[84406]: osd.7 [v2:172.21.15.161:6824/4276809814,v1:172.21.15.161:6825/4276809814] boot 2024-06-01T02:12:31.280 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:30 smithi161 ceph-mon[84406]: osdmap e195: 8 total, 8 up, 8 in 2024-06-01T02:12:31.280 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T02:12:31.280 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:31.280 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:12:31.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:30 smithi116 ceph-mon[89502]: Upgrade: Updating rgw.foo.smithi116.vzjrxj (1/4) 2024-06-01T02:12:31.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:30 smithi116 ceph-mon[89502]: Deploying daemon rgw.foo.smithi116.vzjrxj on smithi116 2024-06-01T02:12:31.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:30 smithi116 ceph-mon[89502]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:12:31.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:30 smithi116 ceph-mon[89502]: osd.7 [v2:172.21.15.161:6824/4276809814,v1:172.21.15.161:6825/4276809814] boot 2024-06-01T02:12:31.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:30 smithi116 ceph-mon[89502]: osdmap e195: 8 total, 8 up, 8 in 2024-06-01T02:12:31.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T02:12:31.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:31.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:12:31.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:30 smithi116 ceph-mon[100606]: Upgrade: Updating rgw.foo.smithi116.vzjrxj (1/4) 2024-06-01T02:12:31.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:30 smithi116 ceph-mon[100606]: Deploying daemon rgw.foo.smithi116.vzjrxj on smithi116 2024-06-01T02:12:31.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:30 smithi116 ceph-mon[100606]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-01T02:12:31.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:30 smithi116 ceph-mon[100606]: osd.7 [v2:172.21.15.161:6824/4276809814,v1:172.21.15.161:6825/4276809814] boot 2024-06-01T02:12:31.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:30 smithi116 ceph-mon[100606]: osdmap e195: 8 total, 8 up, 8 in 2024-06-01T02:12:31.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-01T02:12:31.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:31.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:12:32.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:31 smithi116 ceph-mon[89502]: pgmap v329: 161 pgs: 6 peering, 36 active+undersized, 20 active+undersized+degraded, 99 active+clean; 585 KiB data, 251 MiB used, 715 GiB / 715 GiB avail; 898 B/s rd, 0 op/s; 82/726 objects degraded (11.295%) 2024-06-01T02:12:32.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:31 smithi116 ceph-mon[89502]: osdmap e196: 8 total, 8 up, 8 in 2024-06-01T02:12:32.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:31 smithi116 ceph-mon[89502]: Health check update: Degraded data redundancy: 82/726 objects degraded (11.295%), 20 pgs degraded (PG_DEGRADED) 2024-06-01T02:12:32.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:31 smithi116 ceph-mon[100606]: pgmap v329: 161 pgs: 6 peering, 36 active+undersized, 20 active+undersized+degraded, 99 active+clean; 585 KiB data, 251 MiB used, 715 GiB / 715 GiB avail; 898 B/s rd, 0 op/s; 82/726 objects degraded (11.295%) 2024-06-01T02:12:32.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:31 smithi116 ceph-mon[100606]: osdmap e196: 8 total, 8 up, 8 in 2024-06-01T02:12:32.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:31 smithi116 ceph-mon[100606]: Health check update: Degraded data redundancy: 82/726 objects degraded (11.295%), 20 pgs degraded (PG_DEGRADED) 2024-06-01T02:12:32.294 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:31 smithi161 ceph-mon[84406]: pgmap v329: 161 pgs: 6 peering, 36 active+undersized, 20 active+undersized+degraded, 99 active+clean; 585 KiB data, 251 MiB used, 715 GiB / 715 GiB avail; 898 B/s rd, 0 op/s; 82/726 objects degraded (11.295%) 2024-06-01T02:12:32.294 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:31 smithi161 ceph-mon[84406]: osdmap e196: 8 total, 8 up, 8 in 2024-06-01T02:12:32.294 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:31 smithi161 ceph-mon[84406]: Health check update: Degraded data redundancy: 82/726 objects degraded (11.295%), 20 pgs degraded (PG_DEGRADED) 2024-06-01T02:12:33.165 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:33 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:12:33.166 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:33 smithi161 ceph-mon[84406]: pgmap v331: 161 pgs: 6 peering, 17 active+undersized, 9 active+undersized+degraded, 129 active+clean; 585 KiB data, 255 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 41/726 objects degraded (5.647%) 2024-06-01T02:12:33.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:33 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:12:33.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:33 smithi116 ceph-mon[100606]: pgmap v331: 161 pgs: 6 peering, 17 active+undersized, 9 active+undersized+degraded, 129 active+clean; 585 KiB data, 255 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 41/726 objects degraded (5.647%) 2024-06-01T02:12:33.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:33 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:12:33.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:33 smithi116 ceph-mon[89502]: pgmap v331: 161 pgs: 6 peering, 17 active+undersized, 9 active+undersized+degraded, 129 active+clean; 585 KiB data, 255 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 41/726 objects degraded (5.647%) 2024-06-01T02:12:34.980 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:34.980 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:34.980 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:34 smithi116 ceph-mon[89502]: pgmap v332: 161 pgs: 6 peering, 17 active+undersized, 9 active+undersized+degraded, 129 active+clean; 585 KiB data, 255 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 41/726 objects degraded (5.647%) 2024-06-01T02:12:34.980 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:34 smithi116 ceph-mon[89502]: pgmap v333: 161 pgs: 6 peering, 17 active+undersized, 9 active+undersized+degraded, 129 active+clean; 585 KiB data, 255 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 41/726 objects degraded (5.647%) 2024-06-01T02:12:34.981 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:34.981 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:34.981 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:34.981 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:34.981 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:34.981 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:34 smithi116 ceph-mon[100606]: pgmap v332: 161 pgs: 6 peering, 17 active+undersized, 9 active+undersized+degraded, 129 active+clean; 585 KiB data, 255 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 41/726 objects degraded (5.647%) 2024-06-01T02:12:34.981 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:34 smithi116 ceph-mon[100606]: pgmap v333: 161 pgs: 6 peering, 17 active+undersized, 9 active+undersized+degraded, 129 active+clean; 585 KiB data, 255 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 41/726 objects degraded (5.647%) 2024-06-01T02:12:34.981 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:34.981 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:34.981 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:35.193 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:35.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:35.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:34 smithi161 ceph-mon[84406]: pgmap v332: 161 pgs: 6 peering, 17 active+undersized, 9 active+undersized+degraded, 129 active+clean; 585 KiB data, 255 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 41/726 objects degraded (5.647%) 2024-06-01T02:12:35.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:34 smithi161 ceph-mon[84406]: pgmap v333: 161 pgs: 6 peering, 17 active+undersized, 9 active+undersized+degraded, 129 active+clean; 585 KiB data, 255 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 41/726 objects degraded (5.647%) 2024-06-01T02:12:35.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:35.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:35.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:36.792 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:36.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:36.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:36 smithi116 ceph-mon[89502]: pgmap v334: 161 pgs: 6 peering, 155 active+clean; 585 KiB data, 255 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 23 op/s 2024-06-01T02:12:36.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:36.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:36 smithi116 ceph-mon[89502]: Upgrade: Updating rgw.smpl.smithi116.hokbmf (2/4) 2024-06-01T02:12:36.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:36.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi116.hokbmf", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T02:12:36.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:36.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:36 smithi116 ceph-mon[89502]: Deploying daemon rgw.smpl.smithi116.hokbmf on smithi116 2024-06-01T02:12:36.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:36.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:36.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:36.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:36.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:36.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:36 smithi116 ceph-mon[100606]: pgmap v334: 161 pgs: 6 peering, 155 active+clean; 585 KiB data, 255 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 23 op/s 2024-06-01T02:12:36.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:36.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:36 smithi116 ceph-mon[100606]: Upgrade: Updating rgw.smpl.smithi116.hokbmf (2/4) 2024-06-01T02:12:36.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:36.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi116.hokbmf", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T02:12:36.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:36.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:36 smithi116 ceph-mon[100606]: Deploying daemon rgw.smpl.smithi116.hokbmf on smithi116 2024-06-01T02:12:36.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:36.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:36.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:36.797 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:36.797 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:36.797 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:36 smithi161 ceph-mon[84406]: pgmap v334: 161 pgs: 6 peering, 155 active+clean; 585 KiB data, 255 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 23 op/s 2024-06-01T02:12:36.797 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:36.797 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:36 smithi161 ceph-mon[84406]: Upgrade: Updating rgw.smpl.smithi116.hokbmf (2/4) 2024-06-01T02:12:36.797 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:36.797 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi116.hokbmf", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T02:12:36.797 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:36.797 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:36 smithi161 ceph-mon[84406]: Deploying daemon rgw.smpl.smithi116.hokbmf on smithi116 2024-06-01T02:12:36.798 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:36.798 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:36.798 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:37.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:12:36 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:12:36] "GET /metrics HTTP/1.1" 200 38491 "" "Prometheus/2.33.4" 2024-06-01T02:12:37.773 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:37 smithi116 ceph-mon[100606]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-06-01T02:12:37.774 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:37 smithi116 ceph-mon[100606]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 41/726 objects degraded (5.647%), 9 pgs degraded) 2024-06-01T02:12:37.774 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:37 smithi116 ceph-mon[100606]: Cluster is now healthy 2024-06-01T02:12:37.774 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:37 smithi116 ceph-mon[89502]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-06-01T02:12:37.774 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:37 smithi116 ceph-mon[89502]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 41/726 objects degraded (5.647%), 9 pgs degraded) 2024-06-01T02:12:37.774 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:37 smithi116 ceph-mon[89502]: Cluster is now healthy 2024-06-01T02:12:37.943 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:37 smithi161 ceph-mon[84406]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-06-01T02:12:37.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:37 smithi161 ceph-mon[84406]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 41/726 objects degraded (5.647%), 9 pgs degraded) 2024-06-01T02:12:37.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:37 smithi161 ceph-mon[84406]: Cluster is now healthy 2024-06-01T02:12:38.943 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:38 smithi161 ceph-mon[84406]: pgmap v335: 161 pgs: 161 active+clean; 585 KiB data, 264 MiB used, 715 GiB / 715 GiB avail; 102 KiB/s rd, 154 B/s wr, 159 op/s 2024-06-01T02:12:38.955 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:38 smithi116 ceph-mon[89502]: pgmap v335: 161 pgs: 161 active+clean; 585 KiB data, 264 MiB used, 715 GiB / 715 GiB avail; 102 KiB/s rd, 154 B/s wr, 159 op/s 2024-06-01T02:12:38.955 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:38 smithi116 ceph-mon[100606]: pgmap v335: 161 pgs: 161 active+clean; 585 KiB data, 264 MiB used, 715 GiB / 715 GiB avail; 102 KiB/s rd, 154 B/s wr, 159 op/s 2024-06-01T02:12:40.919 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:40 smithi161 ceph-mon[84406]: pgmap v336: 161 pgs: 161 active+clean; 585 KiB data, 268 MiB used, 715 GiB / 715 GiB avail; 117 KiB/s rd, 127 B/s wr, 185 op/s 2024-06-01T02:12:40.920 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:40.920 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:40 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:41.042 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:40 smithi116 ceph-mon[100606]: pgmap v336: 161 pgs: 161 active+clean; 585 KiB data, 268 MiB used, 715 GiB / 715 GiB avail; 117 KiB/s rd, 127 B/s wr, 185 op/s 2024-06-01T02:12:41.042 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:41.042 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:40 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:41.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:40 smithi116 ceph-mon[89502]: pgmap v336: 161 pgs: 161 active+clean; 585 KiB data, 268 MiB used, 715 GiB / 715 GiB avail; 117 KiB/s rd, 127 B/s wr, 185 op/s 2024-06-01T02:12:41.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:41.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:40 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:42.913 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:42 smithi161 ceph-mon[84406]: pgmap v337: 161 pgs: 161 active+clean; 585 KiB data, 268 MiB used, 715 GiB / 715 GiB avail; 127 KiB/s rd, 118 B/s wr, 200 op/s 2024-06-01T02:12:42.913 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:42.913 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi161.ashqxz", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T02:12:42.913 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:42 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:43.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:42 smithi116 ceph-mon[89502]: pgmap v337: 161 pgs: 161 active+clean; 585 KiB data, 268 MiB used, 715 GiB / 715 GiB avail; 127 KiB/s rd, 118 B/s wr, 200 op/s 2024-06-01T02:12:43.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:43.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi161.ashqxz", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T02:12:43.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:42 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:43.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:42 smithi116 ceph-mon[100606]: pgmap v337: 161 pgs: 161 active+clean; 585 KiB data, 268 MiB used, 715 GiB / 715 GiB avail; 127 KiB/s rd, 118 B/s wr, 200 op/s 2024-06-01T02:12:43.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:43.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi161.ashqxz", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T02:12:43.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:42 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:43.792 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:43 smithi116 ceph-mon[100606]: Upgrade: Updating rgw.foo.smithi161.ashqxz (3/4) 2024-06-01T02:12:43.792 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:43 smithi116 ceph-mon[100606]: Deploying daemon rgw.foo.smithi161.ashqxz on smithi161 2024-06-01T02:12:43.792 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:43 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:12:43.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:43 smithi116 ceph-mon[89502]: Upgrade: Updating rgw.foo.smithi161.ashqxz (3/4) 2024-06-01T02:12:43.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:43 smithi116 ceph-mon[89502]: Deploying daemon rgw.foo.smithi161.ashqxz on smithi161 2024-06-01T02:12:43.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:43 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:12:43.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:43 smithi161 ceph-mon[84406]: Upgrade: Updating rgw.foo.smithi161.ashqxz (3/4) 2024-06-01T02:12:43.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:43 smithi161 ceph-mon[84406]: Deploying daemon rgw.foo.smithi161.ashqxz on smithi161 2024-06-01T02:12:43.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:43 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:12:44.671 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:44 smithi161 ceph-mon[84406]: pgmap v338: 161 pgs: 161 active+clean; 585 KiB data, 268 MiB used, 715 GiB / 715 GiB avail; 177 KiB/s rd, 211 B/s wr, 279 op/s 2024-06-01T02:12:45.026 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:44 smithi116 ceph-mon[100606]: pgmap v338: 161 pgs: 161 active+clean; 585 KiB data, 268 MiB used, 715 GiB / 715 GiB avail; 177 KiB/s rd, 211 B/s wr, 279 op/s 2024-06-01T02:12:45.027 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:44 smithi116 ceph-mon[89502]: pgmap v338: 161 pgs: 161 active+clean; 585 KiB data, 268 MiB used, 715 GiB / 715 GiB avail; 177 KiB/s rd, 211 B/s wr, 279 op/s 2024-06-01T02:12:45.475 INFO:teuthology.orchestra.run.smithi116.stdout:true 2024-06-01T02:12:46.225 INFO:teuthology.orchestra.run.smithi116.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-01T02:12:46.225 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi116 smithi116 running 12s ago 21m - - 2024-06-01T02:12:46.225 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi161 smithi161 running 10s ago 21m - - 2024-06-01T02:12:46.225 INFO:teuthology.orchestra.run.smithi116.stdout:alertmanager.a smithi116 *:9093,9094 running (10m) 12s ago 15m 22.4M - 0.23.0 ba2b418f427c e0c297162ded 2024-06-01T02:12:46.225 INFO:teuthology.orchestra.run.smithi116.stdout:grafana.a smithi161 *:3000 running (10m) 10s ago 15m 53.1M - 8.3.5 dad864ee21e9 00b0a78ee88e 2024-06-01T02:12:46.225 INFO:teuthology.orchestra.run.smithi116.stdout:iscsi.foo.smithi116.gjuxzb smithi116 running (9m) 12s ago 14m 62.9M - 3.5 e1d6a67b021e 0e191fe3e3bb 2024-06-01T02:12:46.225 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.x smithi161 *:8443,9283,8765 running (9m) 10s ago 20m 539M - 19.0.0-2946-g3d89accf b045ac545f82 2d7fbb85e5f7 2024-06-01T02:12:46.225 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.y smithi116 *:8443,9283,8765 running (8m) 12s ago 22m 456M - 19.0.0-2946-g3d89accf b045ac545f82 bc9627c27a57 2024-06-01T02:12:46.226 INFO:teuthology.orchestra.run.smithi116.stdout:mon.a smithi116 running (8m) 12s ago 22m 76.3M 2048M 19.0.0-2946-g3d89accf b045ac545f82 b5e9b6f873ff 2024-06-01T02:12:46.226 INFO:teuthology.orchestra.run.smithi116.stdout:mon.b smithi161 running (6m) 10s ago 20m 51.8M 2048M 19.0.0-2946-g3d89accf b045ac545f82 aaa08c03cfc8 2024-06-01T02:12:46.226 INFO:teuthology.orchestra.run.smithi116.stdout:mon.c smithi116 running (7m) 12s ago 21m 58.2M 2048M 19.0.0-2946-g3d89accf b045ac545f82 bd0fb0e603fa 2024-06-01T02:12:46.226 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.a smithi116 *:9100 running (10m) 12s ago 16m 17.7M - 1.3.1 1dbe0e931976 e347748d17bb 2024-06-01T02:12:46.226 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.b smithi161 *:9100 running (10m) 10s ago 16m 17.3M - 1.3.1 1dbe0e931976 6e66182f5a1a 2024-06-01T02:12:46.226 INFO:teuthology.orchestra.run.smithi116.stdout:osd.0 smithi116 running (5m) 12s ago 20m 55.2M 4096M 19.0.0-2946-g3d89accf b045ac545f82 b4313d121b9d 2024-06-01T02:12:46.226 INFO:teuthology.orchestra.run.smithi116.stdout:osd.1 smithi116 running (5m) 12s ago 19m 57.7M 4096M 19.0.0-2946-g3d89accf b045ac545f82 dccdbeac5aae 2024-06-01T02:12:46.226 INFO:teuthology.orchestra.run.smithi116.stdout:osd.2 smithi116 running (4m) 12s ago 19m 52.1M 4096M 19.0.0-2946-g3d89accf b045ac545f82 1ad95efda404 2024-06-01T02:12:46.226 INFO:teuthology.orchestra.run.smithi116.stdout:osd.3 smithi116 running (4m) 12s ago 19m 55.7M 4096M 19.0.0-2946-g3d89accf b045ac545f82 bd00ad2c2556 2024-06-01T02:12:46.226 INFO:teuthology.orchestra.run.smithi116.stdout:osd.4 smithi161 running (3m) 10s ago 18m 56.1M 4096M 19.0.0-2946-g3d89accf b045ac545f82 0084b9edf7a0 2024-06-01T02:12:46.226 INFO:teuthology.orchestra.run.smithi116.stdout:osd.5 smithi161 running (2m) 10s ago 18m 52.1M 4096M 19.0.0-2946-g3d89accf b045ac545f82 b3849cdeb707 2024-06-01T02:12:46.226 INFO:teuthology.orchestra.run.smithi116.stdout:osd.6 smithi161 running (85s) 10s ago 17m 46.7M 4096M 19.0.0-2946-g3d89accf b045ac545f82 6b87d9299c8d 2024-06-01T02:12:46.226 INFO:teuthology.orchestra.run.smithi116.stdout:osd.7 smithi161 running (22s) 10s ago 17m 50.9M 4096M 19.0.0-2946-g3d89accf b045ac545f82 400bb59c4249 2024-06-01T02:12:46.227 INFO:teuthology.orchestra.run.smithi116.stdout:prometheus.a smithi161 *:9095 running (9m) 10s ago 16m 79.1M - 2.33.4 514e6a882f6e e977f6ca1d73 2024-06-01T02:12:46.227 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi116.vzjrxj smithi116 *:8000 starting - - - - 2024-06-01T02:12:46.227 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi161.ashqxz smithi161 *:8000 starting - - - - 2024-06-01T02:12:46.227 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi116.hokbmf smithi116 *:80 starting - - - - 2024-06-01T02:12:46.227 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi161.axjlzw smithi161 *:80 running (14m) 10s ago 14m 78.0M - 17.2.0 e1d6a67b021e 0b5d5ce47772 2024-06-01T02:12:46.633 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:12:46.633 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": { 2024-06-01T02:12:46.633 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 3 2024-06-01T02:12:46.633 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:12:46.633 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": { 2024-06-01T02:12:46.633 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-01T02:12:46.633 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:12:46.634 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": { 2024-06-01T02:12:46.634 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 8 2024-06-01T02:12:46.634 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:12:46.634 INFO:teuthology.orchestra.run.smithi116.stdout: "rgw": { 2024-06-01T02:12:46.634 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 1, 2024-06-01T02:12:46.634 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-01T02:12:46.634 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:12:46.634 INFO:teuthology.orchestra.run.smithi116.stdout: "overall": { 2024-06-01T02:12:46.634 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 1, 2024-06-01T02:12:46.634 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 15 2024-06-01T02:12:46.634 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T02:12:46.634 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:12:46.937 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:12:46 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:12:46] "GET /metrics HTTP/1.1" 200 38628 "" "Prometheus/2.33.4" 2024-06-01T02:12:46.938 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:46 smithi161 ceph-mon[84406]: from='client.44718 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:12:46.938 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:46 smithi161 ceph-mon[84406]: pgmap v339: 161 pgs: 161 active+clean; 585 KiB data, 268 MiB used, 715 GiB / 715 GiB avail; 161 KiB/s rd, 175 B/s wr, 254 op/s 2024-06-01T02:12:46.938 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:46 smithi161 ceph-mon[84406]: from='client.44724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:12:46.938 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:46.938 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:12:46.938 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:46.938 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:46.938 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:46.938 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:46 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/154985019' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:46.975 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:12:46.976 INFO:teuthology.orchestra.run.smithi116.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-01T02:12:46.976 INFO:teuthology.orchestra.run.smithi116.stdout: "in_progress": true, 2024-06-01T02:12:46.976 INFO:teuthology.orchestra.run.smithi116.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-01T02:12:46.976 INFO:teuthology.orchestra.run.smithi116.stdout: "services_complete": [ 2024-06-01T02:12:46.976 INFO:teuthology.orchestra.run.smithi116.stdout: "mon", 2024-06-01T02:12:46.976 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr", 2024-06-01T02:12:46.976 INFO:teuthology.orchestra.run.smithi116.stdout: "osd" 2024-06-01T02:12:46.976 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-06-01T02:12:46.976 INFO:teuthology.orchestra.run.smithi116.stdout: "progress": "13/23 daemons upgraded", 2024-06-01T02:12:46.976 INFO:teuthology.orchestra.run.smithi116.stdout: "message": "Currently upgrading rgw daemons", 2024-06-01T02:12:46.976 INFO:teuthology.orchestra.run.smithi116.stdout: "is_paused": false 2024-06-01T02:12:46.976 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:12:46.977 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:46 smithi116 ceph-mon[89502]: from='client.44718 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:12:46.977 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:46 smithi116 ceph-mon[89502]: pgmap v339: 161 pgs: 161 active+clean; 585 KiB data, 268 MiB used, 715 GiB / 715 GiB avail; 161 KiB/s rd, 175 B/s wr, 254 op/s 2024-06-01T02:12:46.977 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:46 smithi116 ceph-mon[89502]: from='client.44724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:12:46.977 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:46.977 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:12:46.977 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:46.977 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:46.977 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:46.977 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:46 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/154985019' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:46.978 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:46 smithi116 ceph-mon[100606]: from='client.44718 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:12:46.978 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:46 smithi116 ceph-mon[100606]: pgmap v339: 161 pgs: 161 active+clean; 585 KiB data, 268 MiB used, 715 GiB / 715 GiB avail; 161 KiB/s rd, 175 B/s wr, 254 op/s 2024-06-01T02:12:46.978 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:46 smithi116 ceph-mon[100606]: from='client.44724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:12:46.978 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:46.978 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:12:46.978 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:46.978 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:46.978 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:46.978 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:46 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/154985019' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:47.374 INFO:teuthology.orchestra.run.smithi116.stdout:HEALTH_OK 2024-06-01T02:12:47.851 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:47 smithi161 ceph-mon[84406]: from='client.44748 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:12:47.851 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:47 smithi161 ceph-mon[84406]: from='client.64723 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:12:47.851 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:47 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/1539922234' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:12:47.851 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:47.851 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi161.axjlzw", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T02:12:47.851 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:47 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:48.291 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:47 smithi116 ceph-mon[89502]: from='client.44748 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:12:48.291 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:47 smithi116 ceph-mon[89502]: from='client.64723 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:12:48.291 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:47 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/1539922234' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:12:48.291 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:48.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi161.axjlzw", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T02:12:48.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:47 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:48.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:47 smithi116 ceph-mon[100606]: from='client.44748 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:12:48.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:47 smithi116 ceph-mon[100606]: from='client.64723 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:12:48.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:47 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/1539922234' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:12:48.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:48.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi161.axjlzw", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T02:12:48.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:47 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:49.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:48 smithi116 ceph-mon[89502]: pgmap v340: 161 pgs: 161 active+clean; 585 KiB data, 272 MiB used, 715 GiB / 715 GiB avail; 190 KiB/s rd, 255 B/s wr, 300 op/s 2024-06-01T02:12:49.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:48 smithi116 ceph-mon[89502]: Upgrade: Updating rgw.smpl.smithi161.axjlzw (4/4) 2024-06-01T02:12:49.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:48 smithi116 ceph-mon[89502]: Deploying daemon rgw.smpl.smithi161.axjlzw on smithi161 2024-06-01T02:12:49.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:48 smithi116 ceph-mon[100606]: pgmap v340: 161 pgs: 161 active+clean; 585 KiB data, 272 MiB used, 715 GiB / 715 GiB avail; 190 KiB/s rd, 255 B/s wr, 300 op/s 2024-06-01T02:12:49.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:48 smithi116 ceph-mon[100606]: Upgrade: Updating rgw.smpl.smithi161.axjlzw (4/4) 2024-06-01T02:12:49.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:48 smithi116 ceph-mon[100606]: Deploying daemon rgw.smpl.smithi161.axjlzw on smithi161 2024-06-01T02:12:49.140 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:48 smithi161 ceph-mon[84406]: pgmap v340: 161 pgs: 161 active+clean; 585 KiB data, 272 MiB used, 715 GiB / 715 GiB avail; 190 KiB/s rd, 255 B/s wr, 300 op/s 2024-06-01T02:12:49.141 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:48 smithi161 ceph-mon[84406]: Upgrade: Updating rgw.smpl.smithi161.axjlzw (4/4) 2024-06-01T02:12:49.141 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:48 smithi161 ceph-mon[84406]: Deploying daemon rgw.smpl.smithi161.axjlzw on smithi161 2024-06-01T02:12:51.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:50 smithi116 ceph-mon[89502]: pgmap v341: 161 pgs: 161 active+clean; 585 KiB data, 276 MiB used, 715 GiB / 715 GiB avail; 165 KiB/s rd, 170 B/s wr, 260 op/s 2024-06-01T02:12:51.042 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:50 smithi116 ceph-mon[100606]: pgmap v341: 161 pgs: 161 active+clean; 585 KiB data, 276 MiB used, 715 GiB / 715 GiB avail; 165 KiB/s rd, 170 B/s wr, 260 op/s 2024-06-01T02:12:51.070 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:50 smithi161 ceph-mon[84406]: pgmap v341: 161 pgs: 161 active+clean; 585 KiB data, 276 MiB used, 715 GiB / 715 GiB avail; 165 KiB/s rd, 170 B/s wr, 260 op/s 2024-06-01T02:12:52.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:52 smithi116 ceph-mon[89502]: pgmap v342: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 157 KiB/s rd, 170 B/s wr, 248 op/s 2024-06-01T02:12:52.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:52.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:52.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:52.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:52 smithi116 ceph-mon[100606]: pgmap v342: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 157 KiB/s rd, 170 B/s wr, 248 op/s 2024-06-01T02:12:52.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:52.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:52.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:52.622 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:52 smithi161 ceph-mon[84406]: pgmap v342: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 157 KiB/s rd, 170 B/s wr, 248 op/s 2024-06-01T02:12:52.622 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:52.622 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:52.623 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:53.614 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:53 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:12:53.792 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:53 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:12:53.792 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:53 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:12:54.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:54.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:54.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:54.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:54.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:54.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:54 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:54.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:54 smithi161 ceph-mon[84406]: pgmap v343: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s rd, 170 B/s wr, 227 op/s 2024-06-01T02:12:54.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:54.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:54.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:54.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:54.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:54 smithi161 ceph-mon[84406]: Checking dashboard <-> RGW credentials 2024-06-01T02:12:54.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:54.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:54 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:54.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:54.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:54.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:54.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:54.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:54.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:54.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:54.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:54.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:54.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:54.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:54.600 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:54.600 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:54.600 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:54.600 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:54 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:54.792 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:54.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:54.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:54.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:54.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:54.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:54.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[100606]: pgmap v343: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s rd, 170 B/s wr, 227 op/s 2024-06-01T02:12:54.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:54.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:54.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:54.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:54.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[100606]: Checking dashboard <-> RGW credentials 2024-06-01T02:12:54.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:54.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:54.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:54.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:54.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:54.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:54.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:54.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:54.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:54.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:54.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:54.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:54.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:54.794 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:54.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:54.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:54.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:54.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:54.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:54.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:54.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:54.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:54.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:54.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[89502]: pgmap v343: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s rd, 170 B/s wr, 227 op/s 2024-06-01T02:12:54.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:54.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:54.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:54.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:54.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[89502]: Checking dashboard <-> RGW credentials 2024-06-01T02:12:54.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:54.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:54.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:54.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:54.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:54.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:54.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:54.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:54.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:54.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:54.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:54.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:54.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:54.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:54.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:54.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:54.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:54 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:55.551 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:55.551 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:55.551 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:55.551 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:55.551 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:55.551 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:55.552 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:55.552 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:55.552 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:55.552 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:55.552 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:55.552 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:55.552 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:55.552 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:55.552 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:55.552 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:55.552 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:55.552 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:55.552 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:55.552 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:55.552 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:55.553 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:55.553 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:55.553 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:55.553 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:55.553 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:55.553 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:55.553 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:55.553 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:55.553 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:55.553 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:55.553 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:55.553 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:55.553 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:55.554 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:55.554 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:55.554 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:55.554 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:55.554 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:55.554 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:55.554 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:55.554 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:55 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:55.792 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:55.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:55.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:55.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:55.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:55.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:55.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:55.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:55.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:55.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:55.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:55.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:55.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:55.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:55.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:55.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:55.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:55.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:55.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:55.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:55.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:55.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:55.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:55.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:55.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:55.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:55.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:55.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:55.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:55.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:55.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:55.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:55.795 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:55.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:55.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:55.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:55.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:55.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:55.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:55.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:55.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:55.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:55.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:55.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:55.796 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:55.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:55.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:55.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:55.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:55.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:55.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:55.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:55.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:55.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:55.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:55.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:55.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:55.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:55.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:55.797 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:55.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:55.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:55.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:55.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:55.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:55.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:55.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:55.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:55.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:55.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:55.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:55.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:55.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:55.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:55.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:55.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:55.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:55.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:55.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:55.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:55.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:55.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:55.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:55.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:55 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:56.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:56.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:56.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:56.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:56.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:56.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:56.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:56.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:56.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:56.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: pgmap v344: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 110 KiB/s rd, 85 B/s wr, 173 op/s 2024-06-01T02:12:56.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:56.695 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:56.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:56.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:56.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:56.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:56.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:56.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:56.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:56.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:56.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:56.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:56.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:56.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:56.696 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:56.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:56.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:56.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:56.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:56.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:56.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:56.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:56.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:56.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:56.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:56.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:56.697 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:56.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:56.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:56.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:56.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:56.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:56.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:56.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:56.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:56.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:56.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: pgmap v344: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 110 KiB/s rd, 85 B/s wr, 173 op/s 2024-06-01T02:12:56.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:56.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:56.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:56.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:56.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:56.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:56.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:56.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:56.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:56.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:56.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:56.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:56.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:56.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:56.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:56.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:56.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:56.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:56.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:56.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:56.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:56.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:56.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:56.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:56.798 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:56.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:56.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:56.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:56.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:56.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: pgmap v344: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 110 KiB/s rd, 85 B/s wr, 173 op/s 2024-06-01T02:12:56.799 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:56.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:56.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:56.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:56.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:56.800 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:56.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:56.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:56.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:56.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:56.801 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:56.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:56.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:56.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:56.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:56.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:12:56.802 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:56.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:12:56.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:56.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:56.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:56.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:56.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:56.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:56.803 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:56.804 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:56.804 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:56.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:56.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:12:56.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:56.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:56.805 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:56.805 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:56.805 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:12:57.163 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:12:56 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:12:56] "GET /metrics HTTP/1.1" 200 38628 "" "Prometheus/2.33.4" 2024-06-01T02:12:58.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:58 smithi161 ceph-mon[84406]: pgmap v345: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 147 KiB/s rd, 170 B/s wr, 230 op/s 2024-06-01T02:12:59.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:58 smithi116 ceph-mon[89502]: pgmap v345: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 147 KiB/s rd, 170 B/s wr, 230 op/s 2024-06-01T02:12:59.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:58 smithi116 ceph-mon[100606]: pgmap v345: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 147 KiB/s rd, 170 B/s wr, 230 op/s 2024-06-01T02:12:59.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:59 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi161.ashqxz", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T02:12:59.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:59 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:59.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:59 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:59.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:59 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:59.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:12:59 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:59.982 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:59 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi161.ashqxz", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T02:12:59.983 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:59 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:59.983 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:59 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:59.983 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:59 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:59.983 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:12:59 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:59.983 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi161.ashqxz", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T02:12:59.983 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:12:59.984 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:59.984 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:12:59.984 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:12:59 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:01.042 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:00 smithi116 ceph-mon[100606]: Upgrade: Updating rgw.foo.smithi161.ashqxz (1/2) 2024-06-01T02:13:01.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:00 smithi116 ceph-mon[100606]: Deploying daemon rgw.foo.smithi161.ashqxz on smithi161 2024-06-01T02:13:01.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:00 smithi116 ceph-mon[100606]: pgmap v346: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 120 KiB/s rd, 85 B/s wr, 188 op/s 2024-06-01T02:13:01.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:00 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:01.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:00 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:01.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:00 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:01.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:00 smithi116 ceph-mon[89502]: Upgrade: Updating rgw.foo.smithi161.ashqxz (1/2) 2024-06-01T02:13:01.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:00 smithi116 ceph-mon[89502]: Deploying daemon rgw.foo.smithi161.ashqxz on smithi161 2024-06-01T02:13:01.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:00 smithi116 ceph-mon[89502]: pgmap v346: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 120 KiB/s rd, 85 B/s wr, 188 op/s 2024-06-01T02:13:01.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:01.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:01.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:01.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:00 smithi161 ceph-mon[84406]: Upgrade: Updating rgw.foo.smithi161.ashqxz (1/2) 2024-06-01T02:13:01.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:00 smithi161 ceph-mon[84406]: Deploying daemon rgw.foo.smithi161.ashqxz on smithi161 2024-06-01T02:13:01.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:00 smithi161 ceph-mon[84406]: pgmap v346: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 120 KiB/s rd, 85 B/s wr, 188 op/s 2024-06-01T02:13:01.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:00 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:01.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:00 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:01.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:00 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:01.876 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:01 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:13:02.159 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:13:02.159 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:01 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:13:03.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:02 smithi161 ceph-mon[84406]: pgmap v347: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 95 KiB/s rd, 85 B/s wr, 149 op/s 2024-06-01T02:13:03.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:02 smithi116 ceph-mon[89502]: pgmap v347: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 95 KiB/s rd, 85 B/s wr, 149 op/s 2024-06-01T02:13:03.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:02 smithi116 ceph-mon[100606]: pgmap v347: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 95 KiB/s rd, 85 B/s wr, 149 op/s 2024-06-01T02:13:03.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:03 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:13:04.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:03 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:13:04.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:03 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:13:05.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:04 smithi116 ceph-mon[89502]: pgmap v348: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 81 KiB/s rd, 85 B/s wr, 126 op/s 2024-06-01T02:13:05.042 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:04 smithi116 ceph-mon[100606]: pgmap v348: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 81 KiB/s rd, 85 B/s wr, 126 op/s 2024-06-01T02:13:05.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:04 smithi161 ceph-mon[84406]: pgmap v348: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 81 KiB/s rd, 85 B/s wr, 126 op/s 2024-06-01T02:13:07.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:06 smithi161 ceph-mon[84406]: pgmap v349: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 81 KiB/s rd, 85 B/s wr, 126 op/s 2024-06-01T02:13:07.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:13:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:13:06] "GET /metrics HTTP/1.1" 200 38692 "" "Prometheus/2.33.4" 2024-06-01T02:13:07.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:06 smithi116 ceph-mon[89502]: pgmap v349: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 81 KiB/s rd, 85 B/s wr, 126 op/s 2024-06-01T02:13:07.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:06 smithi116 ceph-mon[100606]: pgmap v349: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 81 KiB/s rd, 85 B/s wr, 126 op/s 2024-06-01T02:13:09.193 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:08 smithi161 ceph-mon[84406]: pgmap v350: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 85 B/s wr, 100 op/s 2024-06-01T02:13:09.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:08 smithi116 ceph-mon[89502]: pgmap v350: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 85 B/s wr, 100 op/s 2024-06-01T02:13:09.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:08 smithi116 ceph-mon[100606]: pgmap v350: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 85 B/s wr, 100 op/s 2024-06-01T02:13:11.146 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:10 smithi116 ceph-mon[89502]: pgmap v351: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 13 KiB/s rd, 0 B/s wr, 20 op/s 2024-06-01T02:13:11.147 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:10 smithi116 ceph-mon[100606]: pgmap v351: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 13 KiB/s rd, 0 B/s wr, 20 op/s 2024-06-01T02:13:11.193 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:10 smithi161 ceph-mon[84406]: pgmap v351: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 13 KiB/s rd, 0 B/s wr, 20 op/s 2024-06-01T02:13:12.542 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:12 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:12.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:12 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:12.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:12 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:12.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:12 smithi116 ceph-mon[89502]: pgmap v352: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:13:12.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:12 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:12.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:12 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:12.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:12 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:12.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:12 smithi116 ceph-mon[100606]: pgmap v352: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:13:12.693 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:12 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:12.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:12 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:12.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:12 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:12.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:12 smithi161 ceph-mon[84406]: pgmap v352: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:13:13.671 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:13 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:13:13.792 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:13 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:13:13.792 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:13 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:13:14.575 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:14 smithi161 ceph-mon[84406]: pgmap v353: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:13:14.792 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:14 smithi116 ceph-mon[89502]: pgmap v353: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:13:14.792 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:14 smithi116 ceph-mon[100606]: pgmap v353: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:13:16.606 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:16.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:16.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:16 smithi161 ceph-mon[84406]: pgmap v354: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-01T02:13:16.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:13:16.792 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:16.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:16.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:16 smithi116 ceph-mon[100606]: pgmap v354: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-01T02:13:16.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:13:16.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:16.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:16.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:16 smithi116 ceph-mon[89502]: pgmap v354: 161 pgs: 161 active+clean; 585 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-01T02:13:16.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:13:16.896 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:13:16 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:13:16] "GET /metrics HTTP/1.1" 200 38676 "" "Prometheus/2.33.4" 2024-06-01T02:13:17.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi161.axjlzw", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T02:13:17.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:17.733 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi161.axjlzw", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T02:13:17.733 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:17.733 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi161.axjlzw", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-01T02:13:17.733 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:17.733 INFO:teuthology.orchestra.run.smithi116.stdout:true 2024-06-01T02:13:18.398 INFO:teuthology.orchestra.run.smithi116.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-01T02:13:18.398 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi116 smithi116 running 6s ago 21m - - 2024-06-01T02:13:18.398 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi161 smithi161 running 18s ago 21m - - 2024-06-01T02:13:18.398 INFO:teuthology.orchestra.run.smithi116.stdout:alertmanager.a smithi116 *:9093,9094 running (11m) 6s ago 16m 22.4M - 0.23.0 ba2b418f427c e0c297162ded 2024-06-01T02:13:18.398 INFO:teuthology.orchestra.run.smithi116.stdout:grafana.a smithi161 *:3000 running (10m) 18s ago 16m 55.7M - 8.3.5 dad864ee21e9 00b0a78ee88e 2024-06-01T02:13:18.398 INFO:teuthology.orchestra.run.smithi116.stdout:iscsi.foo.smithi116.gjuxzb smithi116 running (10m) 6s ago 14m 62.9M - 3.5 e1d6a67b021e 0e191fe3e3bb 2024-06-01T02:13:18.398 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.x smithi161 *:8443,9283,8765 running (10m) 18s ago 21m 541M - 19.0.0-2946-g3d89accf b045ac545f82 2d7fbb85e5f7 2024-06-01T02:13:18.399 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.y smithi116 *:8443,9283,8765 running (9m) 6s ago 22m 456M - 19.0.0-2946-g3d89accf b045ac545f82 bc9627c27a57 2024-06-01T02:13:18.399 INFO:teuthology.orchestra.run.smithi116.stdout:mon.a smithi116 running (8m) 6s ago 22m 79.2M 2048M 19.0.0-2946-g3d89accf b045ac545f82 b5e9b6f873ff 2024-06-01T02:13:18.399 INFO:teuthology.orchestra.run.smithi116.stdout:mon.b smithi161 running (7m) 18s ago 21m 52.7M 2048M 19.0.0-2946-g3d89accf b045ac545f82 aaa08c03cfc8 2024-06-01T02:13:18.399 INFO:teuthology.orchestra.run.smithi116.stdout:mon.c smithi116 running (7m) 6s ago 21m 59.0M 2048M 19.0.0-2946-g3d89accf b045ac545f82 bd0fb0e603fa 2024-06-01T02:13:18.399 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.a smithi116 *:9100 running (11m) 6s ago 17m 17.3M - 1.3.1 1dbe0e931976 e347748d17bb 2024-06-01T02:13:18.399 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.b smithi161 *:9100 running (10m) 18s ago 16m 20.7M - 1.3.1 1dbe0e931976 6e66182f5a1a 2024-06-01T02:13:18.399 INFO:teuthology.orchestra.run.smithi116.stdout:osd.0 smithi116 running (6m) 6s ago 20m 58.7M 4096M 19.0.0-2946-g3d89accf b045ac545f82 b4313d121b9d 2024-06-01T02:13:18.399 INFO:teuthology.orchestra.run.smithi116.stdout:osd.1 smithi116 running (5m) 6s ago 20m 63.4M 4096M 19.0.0-2946-g3d89accf b045ac545f82 dccdbeac5aae 2024-06-01T02:13:18.399 INFO:teuthology.orchestra.run.smithi116.stdout:osd.2 smithi116 running (5m) 6s ago 20m 55.8M 4096M 19.0.0-2946-g3d89accf b045ac545f82 1ad95efda404 2024-06-01T02:13:18.399 INFO:teuthology.orchestra.run.smithi116.stdout:osd.3 smithi116 running (4m) 6s ago 19m 62.1M 4096M 19.0.0-2946-g3d89accf b045ac545f82 bd00ad2c2556 2024-06-01T02:13:18.399 INFO:teuthology.orchestra.run.smithi116.stdout:osd.4 smithi161 running (3m) 18s ago 19m 61.0M 4096M 19.0.0-2946-g3d89accf b045ac545f82 0084b9edf7a0 2024-06-01T02:13:18.399 INFO:teuthology.orchestra.run.smithi116.stdout:osd.5 smithi161 running (2m) 18s ago 18m 55.3M 4096M 19.0.0-2946-g3d89accf b045ac545f82 b3849cdeb707 2024-06-01T02:13:18.399 INFO:teuthology.orchestra.run.smithi116.stdout:osd.6 smithi161 running (117s) 18s ago 18m 50.0M 4096M 19.0.0-2946-g3d89accf b045ac545f82 6b87d9299c8d 2024-06-01T02:13:18.399 INFO:teuthology.orchestra.run.smithi116.stdout:osd.7 smithi161 running (54s) 18s ago 18m 56.6M 4096M 19.0.0-2946-g3d89accf b045ac545f82 400bb59c4249 2024-06-01T02:13:18.400 INFO:teuthology.orchestra.run.smithi116.stdout:prometheus.a smithi161 *:9095 running (9m) 18s ago 17m 79.1M - 2.33.4 514e6a882f6e e977f6ca1d73 2024-06-01T02:13:18.400 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi116.vzjrxj smithi116 *:8000 running (44s) 6s ago 15m 78.0M - 19.0.0-2946-g3d89accf b045ac545f82 376c2daccbd5 2024-06-01T02:13:18.400 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi161.ashqxz smithi161 *:8000 starting - - - - 2024-06-01T02:13:18.400 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi116.hokbmf smithi116 *:80 running (38s) 6s ago 15m 78.0M - 19.0.0-2946-g3d89accf b045ac545f82 3b0ab9880bcf 2024-06-01T02:13:18.400 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi161.axjlzw smithi161 *:80 running (25s) 18s ago 15m 77.7M - 19.0.0-2946-g3d89accf b045ac545f82 87b6bb5dc396 2024-06-01T02:13:18.596 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:18 smithi161 ceph-mon[84406]: Upgrade: Updating rgw.smpl.smithi161.axjlzw (2/2) 2024-06-01T02:13:18.596 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:18 smithi161 ceph-mon[84406]: Deploying daemon rgw.smpl.smithi161.axjlzw on smithi161 2024-06-01T02:13:18.596 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:18 smithi161 ceph-mon[84406]: pgmap v355: 161 pgs: 161 active+clean; 585 KiB data, 292 MiB used, 715 GiB / 715 GiB avail; 51 KiB/s rd, 85 B/s wr, 79 op/s 2024-06-01T02:13:18.596 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:18 smithi161 ceph-mon[84406]: from='client.64810 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:13:18.792 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:18 smithi116 ceph-mon[89502]: Upgrade: Updating rgw.smpl.smithi161.axjlzw (2/2) 2024-06-01T02:13:18.792 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:18 smithi116 ceph-mon[89502]: Deploying daemon rgw.smpl.smithi161.axjlzw on smithi161 2024-06-01T02:13:18.792 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:18 smithi116 ceph-mon[89502]: pgmap v355: 161 pgs: 161 active+clean; 585 KiB data, 292 MiB used, 715 GiB / 715 GiB avail; 51 KiB/s rd, 85 B/s wr, 79 op/s 2024-06-01T02:13:18.792 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:18 smithi116 ceph-mon[89502]: from='client.64810 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:13:18.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:18 smithi116 ceph-mon[100606]: Upgrade: Updating rgw.smpl.smithi161.axjlzw (2/2) 2024-06-01T02:13:18.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:18 smithi116 ceph-mon[100606]: Deploying daemon rgw.smpl.smithi161.axjlzw on smithi161 2024-06-01T02:13:18.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:18 smithi116 ceph-mon[100606]: pgmap v355: 161 pgs: 161 active+clean; 585 KiB data, 292 MiB used, 715 GiB / 715 GiB avail; 51 KiB/s rd, 85 B/s wr, 79 op/s 2024-06-01T02:13:18.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:18 smithi116 ceph-mon[100606]: from='client.64810 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:13:18.801 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:13:18.802 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": { 2024-06-01T02:13:18.802 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 3 2024-06-01T02:13:18.802 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:13:18.802 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": { 2024-06-01T02:13:18.802 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-01T02:13:18.803 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:13:18.803 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": { 2024-06-01T02:13:18.803 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 8 2024-06-01T02:13:18.803 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:13:18.803 INFO:teuthology.orchestra.run.smithi116.stdout: "rgw": { 2024-06-01T02:13:18.803 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 5 2024-06-01T02:13:18.803 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:13:18.803 INFO:teuthology.orchestra.run.smithi116.stdout: "overall": { 2024-06-01T02:13:18.803 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 18 2024-06-01T02:13:18.803 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T02:13:18.803 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:13:19.143 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:13:19.143 INFO:teuthology.orchestra.run.smithi116.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-01T02:13:19.143 INFO:teuthology.orchestra.run.smithi116.stdout: "in_progress": true, 2024-06-01T02:13:19.143 INFO:teuthology.orchestra.run.smithi116.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-01T02:13:19.143 INFO:teuthology.orchestra.run.smithi116.stdout: "services_complete": [ 2024-06-01T02:13:19.143 INFO:teuthology.orchestra.run.smithi116.stdout: "mon", 2024-06-01T02:13:19.143 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr", 2024-06-01T02:13:19.144 INFO:teuthology.orchestra.run.smithi116.stdout: "osd" 2024-06-01T02:13:19.144 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-06-01T02:13:19.144 INFO:teuthology.orchestra.run.smithi116.stdout: "progress": "16/23 daemons upgraded", 2024-06-01T02:13:19.144 INFO:teuthology.orchestra.run.smithi116.stdout: "message": "Currently upgrading rgw daemons", 2024-06-01T02:13:19.144 INFO:teuthology.orchestra.run.smithi116.stdout: "is_paused": false 2024-06-01T02:13:19.144 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:13:19.542 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:19 smithi116 ceph-mon[89502]: from='client.44844 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:13:19.542 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:19 smithi116 ceph-mon[89502]: from='client.64822 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:13:19.542 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:19 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/22552026' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:19.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:19 smithi116 ceph-mon[100606]: from='client.44844 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:13:19.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:19 smithi116 ceph-mon[100606]: from='client.64822 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:13:19.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:19 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/22552026' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:19.556 INFO:teuthology.orchestra.run.smithi116.stdout:HEALTH_OK 2024-06-01T02:13:19.640 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:19 smithi161 ceph-mon[84406]: from='client.44844 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:13:19.640 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:19 smithi161 ceph-mon[84406]: from='client.64822 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:13:19.640 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:19 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/22552026' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:20.620 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:20 smithi161 ceph-mon[84406]: from='client.44862 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:13:20.620 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:20 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/1610298852' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:13:20.620 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:20 smithi161 ceph-mon[84406]: pgmap v356: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 75 KiB/s rd, 85 B/s wr, 118 op/s 2024-06-01T02:13:20.620 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:20.620 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:20.621 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:20 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:20.792 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:20 smithi116 ceph-mon[89502]: from='client.44862 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:13:20.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:20 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/1610298852' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:13:20.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:20 smithi116 ceph-mon[89502]: pgmap v356: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 75 KiB/s rd, 85 B/s wr, 118 op/s 2024-06-01T02:13:20.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:20.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:20.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:20 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:20.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:20 smithi116 ceph-mon[100606]: from='client.44862 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:13:20.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:20 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/1610298852' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:13:20.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:20 smithi116 ceph-mon[100606]: pgmap v356: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 75 KiB/s rd, 85 B/s wr, 118 op/s 2024-06-01T02:13:20.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:20.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:20.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:20 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:22.943 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:22 smithi161 ceph-mon[84406]: pgmap v357: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 79 KiB/s rd, 85 B/s wr, 124 op/s 2024-06-01T02:13:23.042 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:22 smithi116 ceph-mon[100606]: pgmap v357: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 79 KiB/s rd, 85 B/s wr, 124 op/s 2024-06-01T02:13:23.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:22 smithi116 ceph-mon[89502]: pgmap v357: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 79 KiB/s rd, 85 B/s wr, 124 op/s 2024-06-01T02:13:24.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:23 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:13:24.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:24.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:24.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:23 smithi116 ceph-mon[89502]: pgmap v358: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 85 KiB/s rd, 91 B/s wr, 133 op/s 2024-06-01T02:13:24.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:23 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:24.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:23 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:13:24.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:24.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:24.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:23 smithi116 ceph-mon[100606]: pgmap v358: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 85 KiB/s rd, 91 B/s wr, 133 op/s 2024-06-01T02:13:24.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:23 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:24.043 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:23 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:13:24.044 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:24.044 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:24.044 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:23 smithi161 ceph-mon[84406]: pgmap v358: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 85 KiB/s rd, 91 B/s wr, 133 op/s 2024-06-01T02:13:24.044 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:23 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:24.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:24 smithi161 ceph-mon[84406]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-06-01T02:13:25.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:24 smithi116 ceph-mon[89502]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-06-01T02:13:25.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:24 smithi116 ceph-mon[100606]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-06-01T02:13:26.107 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:25 smithi161 ceph-mon[84406]: pgmap v359: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 84 KiB/s rd, 91 B/s wr, 132 op/s 2024-06-01T02:13:26.107 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:26.107 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:26.107 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:26.107 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:26.107 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:26.107 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:26.107 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:26.107 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:26.107 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:25 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:26.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:25 smithi116 ceph-mon[100606]: pgmap v359: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 84 KiB/s rd, 91 B/s wr, 132 op/s 2024-06-01T02:13:26.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:26.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:26.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:26.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:26.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:26.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:26.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:26.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:26.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:25 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:26.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:25 smithi116 ceph-mon[89502]: pgmap v359: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 84 KiB/s rd, 91 B/s wr, 132 op/s 2024-06-01T02:13:26.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:26.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:26.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:26.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:26.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:26.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:26.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:26.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:26.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:25 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:26.943 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:13:26] "GET /metrics HTTP/1.1" 200 38676 "" "Prometheus/2.33.4" 2024-06-01T02:13:26.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: pgmap v360: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 94 KiB/s rd, 102 B/s wr, 148 op/s 2024-06-01T02:13:26.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:26.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: Checking dashboard <-> RGW credentials 2024-06-01T02:13:26.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-06-01T02:13:26.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: Cluster is now healthy 2024-06-01T02:13:26.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:26.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:26.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:26.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:26.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:26.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:26.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:26.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:26.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:26.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:26.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:26.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:26.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:26.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:26.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:26.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:26.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:26.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:26.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:26.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:26.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:26.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:26.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:26.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:26.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:26.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:26.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:26.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:26.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:26.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:26.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:26.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:26 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:27.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: pgmap v360: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 94 KiB/s rd, 102 B/s wr, 148 op/s 2024-06-01T02:13:27.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:27.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: Checking dashboard <-> RGW credentials 2024-06-01T02:13:27.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-06-01T02:13:27.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: Cluster is now healthy 2024-06-01T02:13:27.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:27.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:27.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:27.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:27.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:27.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:27.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:27.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:27.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:27.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:27.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:27.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:27.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:27.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:27.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:27.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:27.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:27.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:27.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:27.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:27.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:27.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:27.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:27.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:27.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: pgmap v360: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 94 KiB/s rd, 102 B/s wr, 148 op/s 2024-06-01T02:13:27.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:27.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: Checking dashboard <-> RGW credentials 2024-06-01T02:13:27.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-06-01T02:13:27.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: Cluster is now healthy 2024-06-01T02:13:27.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:27.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:27.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:27.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:27.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:27.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:27.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:27.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:27.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:27.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:27.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:27.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:27.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:27.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:27.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:27.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:27.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:27.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:27.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:27.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:27.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:27.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:27.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:27.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:27.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:27.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:27.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:27.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:27.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:27.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:27.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:27.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:27.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:27.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:27.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:27.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:27.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:27.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:27.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:27.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:26 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:28.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:28.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:28.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:28.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:28.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:28.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:28.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:28.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:28.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:28.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:28.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:28.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:28.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:28.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:28.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:28.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:28.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:28.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:28.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:28.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:28.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:28.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:28.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:28.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:28.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:28.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:28.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:28.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:28.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:28.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:28.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:28.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:28.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:28.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:28.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:28.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:28.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:28.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:28.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:28.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:28.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:28.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:28.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:28.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:28.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:28.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:28.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:27 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:28.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:28.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:28.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:28.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:28.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:28.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:28.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:28.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:28.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:28.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:28.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:28.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:28.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:28.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:28.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:28.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:28.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:28.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:28.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:28.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:28.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:28.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:28.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:28.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:28.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:28.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:28.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:28.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:28.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:28.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:28.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:28.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:28.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:28.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:28.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:28.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:28.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:28.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:28.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:28.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:28.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:28.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:28.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:28.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:28.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:28.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:28.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:28.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:28.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:28.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:28.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:28.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:28.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:28.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:28.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:28.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:28.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:28.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:28.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:28.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:28.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:28.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:28.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:28.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:28.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:28.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:28.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:28.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:28.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:28.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:28.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:28.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:28.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:28.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:28.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:28.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:28.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:28.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:28.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:28.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:28.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:28.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:28.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:28.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:28.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:28.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:28.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:28.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:28.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:28.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:28.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:28.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:28.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:28.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:27 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:29.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:29.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:29.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:29.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:29.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:29.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: pgmap v361: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 92 KiB/s rd, 102 B/s wr, 144 op/s 2024-06-01T02:13:29.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:29.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:29.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:29.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:29.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:29.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:29.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:29.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:29.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:29.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:29.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:29.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:29.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:29.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:29.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:29.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:29.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:29.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:29.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:29.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:29.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:29.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:29.044 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:29.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:29.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:29.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:29.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:29.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:29.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: pgmap v361: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 92 KiB/s rd, 102 B/s wr, 144 op/s 2024-06-01T02:13:29.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:29.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:29.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:29.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:29.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:29.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:29.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:29.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:29.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:29.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:29.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:29.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:29.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:29.046 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:29.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:29.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:29.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:29.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:29.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:29.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:29.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:29.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:29.047 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:29.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:29.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:29.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:29.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:29.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:29.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:29.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:29.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:29.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:29.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:29.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:29.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:29.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:29.048 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:29.049 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:29.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:29.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:29.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:29.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:29.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:29.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:29.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:29.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:29.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:29.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:29.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:29.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:29.049 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:29.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:29.050 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:28 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:29.118 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:29.118 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:29.118 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:29.118 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:29.119 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:29.119 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: pgmap v361: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 92 KiB/s rd, 102 B/s wr, 144 op/s 2024-06-01T02:13:29.119 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:29.119 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:29.119 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:29.119 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:29.119 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:29.119 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:29.119 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:29.119 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:29.119 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:29.119 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:29.119 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:29.120 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:29.120 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:29.120 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:29.120 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:29.120 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:29.120 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:29.120 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:29.120 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:29.120 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:29.120 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:29.120 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:29.120 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:29.120 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:29.120 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:29.121 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:29.121 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:29.121 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:29.121 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:29.121 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:29.121 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:29.121 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:29.121 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:29.121 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:29.121 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:29.121 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:29.121 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:29.121 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:28 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:29 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:29 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: pgmap v362: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 92 KiB/s rd, 102 B/s wr, 144 op/s 2024-06-01T02:13:30.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:30.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:30.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:30.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:30.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:30.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:30.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:31.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:31.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:31.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:31.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:31.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:31.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:31.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: pgmap v362: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 92 KiB/s rd, 102 B/s wr, 144 op/s 2024-06-01T02:13:31.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:31.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:31.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:31.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:31.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:31.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:31.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:31.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:31.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:31.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:31.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:31.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:31.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:31.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:31.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:31.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:31.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:31.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:31.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:31.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:31.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:31.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:31.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:31.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:31.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:31.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:31.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:31.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:31.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:31.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: pgmap v362: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 92 KiB/s rd, 102 B/s wr, 144 op/s 2024-06-01T02:13:31.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:31.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:31.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:31.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:31.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:31.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:31.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:31.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:31.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:31.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:31.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:31.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:31.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:31.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:31.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:31.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:31.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:31.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:31.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:31.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:31.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:31.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:31.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:31.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:31.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:31.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:31.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:31.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:31.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:31.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:31.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:31.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:31.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:31.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:31.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:31.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:31.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:31.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:31.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:31.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:31.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:31.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:31.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:31.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:31.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:31.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:31.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:31.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:31.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:31.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:31.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:31.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:31.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:31.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:31.301 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:31.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:31.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:31.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:31.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:31.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:31.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:31.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:31.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:31.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:31.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:31.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:31.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:31.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:31.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:31.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:31.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:31.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:31.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:31.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:31.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:31.834 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:13:31.834 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:31.834 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:31.834 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:31.834 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:31.834 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:31.834 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:31.834 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:31.834 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:31.834 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:31.834 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:31.834 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:31.834 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:31.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:31.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:31.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:31.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:31.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:31.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:31.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:31.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:31.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:31.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:31.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:31.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:31.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:31.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:31.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:31.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:31.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:31.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:31.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:31.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:31.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:31.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:31.837 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:31.837 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:31.837 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:31.837 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:31.837 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:32.250 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:32.251 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:32.251 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:32.251 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:32.251 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:32.251 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:32.251 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:32.251 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:32.251 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:32.251 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:32.251 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:32.251 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:32.251 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:13:32.252 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:32.252 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:32.252 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:32.252 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:32.252 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:32.252 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:32.252 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:32.252 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:32.252 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:32.252 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:32.252 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:32.252 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:32.252 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:32.253 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:32.253 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:32.253 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:32.253 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:32.253 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:32.253 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:32.253 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:32.254 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:32.254 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:32.254 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:32.254 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:32.254 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:32.254 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:32.254 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:32.254 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:32.255 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:13:32.255 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:32.255 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:32.255 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:32.255 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:32.255 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:32.255 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:32.255 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:32.255 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:32.255 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:32.255 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:32.255 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:32.255 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:32.255 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:32.256 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:32.256 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:32.256 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:32.256 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:32.256 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:32.256 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:32.256 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:32.256 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:32.256 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:32.257 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:32.257 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:32.257 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:32.257 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:32.257 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:32.257 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:32.257 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:32.257 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:32.257 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:32.257 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:32.257 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:32.257 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:32.257 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:32.257 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:32.258 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:32.258 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:32.258 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:32.258 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:32.258 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:32.258 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:32.258 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:32.258 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:32.258 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:32.258 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:32.259 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:32.259 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:32.259 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:32.259 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:32.259 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:32.259 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:32.259 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:32.259 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:32.260 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:32.260 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:32.260 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:32.260 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:32.260 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:32.260 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:32.260 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:32.260 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:32.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:32.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:32.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:32.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:32.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:32.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: pgmap v363: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 97 KiB/s rd, 102 B/s wr, 151 op/s 2024-06-01T02:13:32.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:32.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:32.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:32.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:32.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:32.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:32.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:32.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:32.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:32.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:32.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:32.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:32.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:32.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:32.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:32.946 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:32.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:32.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:32.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:32.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:32.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:32.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:32.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:32.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:32.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:32.947 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:32.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:32.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:32.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:32.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:32.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:32.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:32.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:32.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:32.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:32.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:32.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:32.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:32.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:32.948 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:32.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:32.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:32.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:32.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:32.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:32.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:32.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:32.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:32.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:32.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:32.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:32.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:32.949 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:32 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:33.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:33.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:33.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:33.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:33.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:33.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: pgmap v363: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 97 KiB/s rd, 102 B/s wr, 151 op/s 2024-06-01T02:13:33.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:33.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:33.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:33.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:33.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:33.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:33.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:33.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:33.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:33.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:33.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:33.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:33.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:33.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:33.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:33.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:33.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:33.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:33.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:33.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:33.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:33.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:33.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:33.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:33.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:33.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:33.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:33.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:33.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: pgmap v363: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 97 KiB/s rd, 102 B/s wr, 151 op/s 2024-06-01T02:13:33.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:33.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:33.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:33.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:33.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:33.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:33.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:33.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:33.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:33.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:33.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:33.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:33.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:33.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:33.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:33.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:33.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:33.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:33.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:33.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:33.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:33.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:33.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:33.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:33.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:33.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:33.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:33.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:33.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:33.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:33.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:33.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:33.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:33.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:33.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:33.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:33.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:33.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:33.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:33.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:33.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:33.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:33.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:33.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:33.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:33.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:33.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:33.302 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:33.302 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:33.302 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:33.302 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:33.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:33.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:33.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:33.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:33.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:33.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:33.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:33.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:33.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:33.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:33.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:33.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:33.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:33.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:33.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:33.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:33.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:33.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:33.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:33.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:33.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:33.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:33.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:33.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:33.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:33.305 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:33.305 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:33.305 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:33.305 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:33.305 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:33.305 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:33.305 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:32 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:34.074 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:34.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:34.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:13:34.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:34.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:34.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:34.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:34.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:34.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:34.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:34.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:34.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:34.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:34.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:34.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:34.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:34.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:34.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:34.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:34.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:34.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:34.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:34.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:34.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:34.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:34.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:34.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:34.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:34.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:34.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:34.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:34.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:34.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:34.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:34.078 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:34.078 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:34.078 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:34.078 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:34.078 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:34.078 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:34.078 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:34.078 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:34.078 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:34.078 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:34.078 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:34.078 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:34.078 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:34.079 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:34.079 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:34.079 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:33 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:34.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:34.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:34.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:13:34.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:34.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:34.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:34.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:34.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:34.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:34.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:34.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:34.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:34.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:34.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:34.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:34.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:34.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:34.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:34.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:34.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:34.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:34.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:34.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:34.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:34.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:34.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:34.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:34.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:34.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:34.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:34.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:34.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:34.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:13:34.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:34.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:34.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:34.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:34.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:34.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:34.296 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:34.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:34.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:34.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:34.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:34.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:34.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:34.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:34.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:34.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:34.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:34.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:34.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:34.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:34.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:34.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:34.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:34.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:34.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:34.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:34.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:34.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:34.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:34.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:34.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:34.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:34.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:34.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:34.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:34.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:34.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:34.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:34.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:34.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:34.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:34.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:34.299 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:34.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:34.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:34.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:34.300 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:34.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:34.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:34.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:34.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:34.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:34.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:34.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:34.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:34.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:34.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:34.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:34.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:34.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:34.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:34.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:34.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:34.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:34.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:34.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:34.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:33 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:35.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:35.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:35.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:35.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:35.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:35.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: pgmap v364: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 90 KiB/s rd, 94 B/s wr, 140 op/s 2024-06-01T02:13:35.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:35.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:35.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:35.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:35.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:35.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:35.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:35.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:35.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:35.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:35.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:35.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:35.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:35.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:35.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:35.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:35.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:35.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:35.078 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:35.078 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:35.078 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:35.078 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:35.078 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:35.078 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:35.078 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:35.078 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:35.078 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:35.078 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:35.078 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:35.078 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:35.078 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:35.079 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:35.079 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:35.079 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:35.079 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:35.079 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:35.079 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:35.079 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:35.079 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:35.079 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:35.079 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:35.079 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:35.079 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:35.079 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:35.079 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:35.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:35.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:35.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:35.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:34 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:35.196 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:35.196 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:35.196 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:35.196 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:35.196 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:35.196 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: pgmap v364: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 90 KiB/s rd, 94 B/s wr, 140 op/s 2024-06-01T02:13:35.196 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:35.196 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:35.197 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:35.197 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:35.197 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:35.197 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:35.197 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:35.197 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:35.197 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:35.197 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:35.197 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:35.197 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:35.197 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:35.197 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:35.197 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:35.198 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:35.198 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:35.198 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:35.198 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:35.198 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:35.198 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:35.198 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:35.198 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:35.198 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:35.199 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:35.199 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:35.199 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:35.199 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:35.199 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:35.199 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: pgmap v364: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 90 KiB/s rd, 94 B/s wr, 140 op/s 2024-06-01T02:13:35.199 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:35.199 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:35.199 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:35.199 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:35.199 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:35.200 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:35.200 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:35.200 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:35.200 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:35.200 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:35.200 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:35.200 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:35.200 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:35.200 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:35.200 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:35.200 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:35.200 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:35.200 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:35.200 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:35.201 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:35.201 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:35.201 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:35.201 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:35.201 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:35.201 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:35.201 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:35.201 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:35.201 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:35.201 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:35.201 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:35.202 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:35.202 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:35.202 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:35.202 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:35.202 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:35.202 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:35.202 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:35.202 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:35.202 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:35.202 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:35.202 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:35.202 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:35.202 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:35.202 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:35.203 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:35.203 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:35.203 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:35.203 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:35.203 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:35.203 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:35.203 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:35.203 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:35.203 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:35.203 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:35.203 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:35.203 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:35.203 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:35.204 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:35.204 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:35.204 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:35.204 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:35.204 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:35.204 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:35.204 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:35.204 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:35.204 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:35.204 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:35.204 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:35.204 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:35.204 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:35.204 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:35.204 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:35.205 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:35.205 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:34 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:36.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:36.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:36.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:36.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:36.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:36.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:36.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:36.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:36.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:36.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:36.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:36.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:36.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:36.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:36.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:36.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:36.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:36.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:36.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:36.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:36.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:36.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:36.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:36.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:36.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:36.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:36.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:36.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:36.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:36.197 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:36.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:36.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:36.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:36.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:36.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:36.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:36.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:36.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:36.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:36.198 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:36.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:36.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:36.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:36.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.199 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:35 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:36.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:36.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:36.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:36.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:36.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:36.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:36.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:36.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:36.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:36.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:36.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:36.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:36.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:36.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:36.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:36.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:36.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:36.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:36.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:36.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:36.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:36.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:36.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:36.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:36.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:36.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:36.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:36.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:36.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:36.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:36.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:36.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:36.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:36.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:36.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:36.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:36.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:36.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:36.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:36.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:36.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:36.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:36.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:36.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:36.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:36.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:36.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:36.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:36.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:36.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:36.297 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:36.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:36.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:36.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:36.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:36.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:36.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:36.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:36.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:36.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:36.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:36.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:36.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:36.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:36.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:36.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:36.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:36.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:36.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:36.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:36.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:36.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:36.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:36.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:36.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:36.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:36.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:36.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:36.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:36.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:36.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:36.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:36.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:36.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:36.300 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:36.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:36.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:36.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:36.301 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:35 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:37.101 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:13:36] "GET /metrics HTTP/1.1" 200 38672 "" "Prometheus/2.33.4" 2024-06-01T02:13:37.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:37.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:37.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:37.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:37.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:37.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:37.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: pgmap v365: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 90 KiB/s rd, 94 B/s wr, 140 op/s 2024-06-01T02:13:37.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:37.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:37.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:37.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:37.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:37.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:37.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:37.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:37.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:37.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: Upgrade: Setting container_image for all rgw 2024-06-01T02:13:37.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:37.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.foo.smithi116.vzjrxj"}]: dispatch 2024-06-01T02:13:37.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.foo.smithi116.vzjrxj"}]': finished 2024-06-01T02:13:37.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.foo.smithi161.ashqxz"}]: dispatch 2024-06-01T02:13:37.103 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.foo.smithi161.ashqxz"}]': finished 2024-06-01T02:13:37.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.smpl.smithi116.hokbmf"}]: dispatch 2024-06-01T02:13:37.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.smpl.smithi116.hokbmf"}]': finished 2024-06-01T02:13:37.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.smpl.smithi161.axjlzw"}]: dispatch 2024-06-01T02:13:37.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.smpl.smithi161.axjlzw"}]': finished 2024-06-01T02:13:37.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:37.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: Upgrade: Setting container_image for all rbd-mirror 2024-06-01T02:13:37.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:37.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:37.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:37.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: Upgrade: Setting container_image for all ceph-exporter 2024-06-01T02:13:37.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:37.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:37.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:37.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:37.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:37.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:37.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:37.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:37.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:37.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:37.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: pgmap v365: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 90 KiB/s rd, 94 B/s wr, 140 op/s 2024-06-01T02:13:37.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:37.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:37.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:37.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:37.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:37.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:37.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:37.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:37.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:37.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: Upgrade: Setting container_image for all rgw 2024-06-01T02:13:37.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:37.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.foo.smithi116.vzjrxj"}]: dispatch 2024-06-01T02:13:37.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.foo.smithi116.vzjrxj"}]': finished 2024-06-01T02:13:37.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.foo.smithi161.ashqxz"}]: dispatch 2024-06-01T02:13:37.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.foo.smithi161.ashqxz"}]': finished 2024-06-01T02:13:37.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.smpl.smithi116.hokbmf"}]: dispatch 2024-06-01T02:13:37.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.smpl.smithi116.hokbmf"}]': finished 2024-06-01T02:13:37.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.smpl.smithi161.axjlzw"}]: dispatch 2024-06-01T02:13:37.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.smpl.smithi161.axjlzw"}]': finished 2024-06-01T02:13:37.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:37.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:37.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:37.295 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:37.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:37.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:37.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:37.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: pgmap v365: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 90 KiB/s rd, 94 B/s wr, 140 op/s 2024-06-01T02:13:37.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:37.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: Metadata not up to date on all hosts. Skipping non agent specs 2024-06-01T02:13:37.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-01T02:13:37.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:37.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:37.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:37.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:37.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:37.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:37.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: Upgrade: Setting container_image for all rgw 2024-06-01T02:13:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.foo.smithi116.vzjrxj"}]: dispatch 2024-06-01T02:13:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.foo.smithi116.vzjrxj"}]': finished 2024-06-01T02:13:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.foo.smithi161.ashqxz"}]: dispatch 2024-06-01T02:13:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.foo.smithi161.ashqxz"}]': finished 2024-06-01T02:13:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.smpl.smithi116.hokbmf"}]: dispatch 2024-06-01T02:13:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.smpl.smithi116.hokbmf"}]': finished 2024-06-01T02:13:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.smpl.smithi161.axjlzw"}]: dispatch 2024-06-01T02:13:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.smpl.smithi161.axjlzw"}]': finished 2024-06-01T02:13:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: Upgrade: Setting container_image for all rbd-mirror 2024-06-01T02:13:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:37.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:37.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:37.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: Upgrade: Setting container_image for all ceph-exporter 2024-06-01T02:13:37.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:37.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:37.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:37.298 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:37.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: Upgrade: Setting container_image for all rbd-mirror 2024-06-01T02:13:37.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:37.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:37.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:37.298 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: Upgrade: Setting container_image for all ceph-exporter 2024-06-01T02:13:37.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:37.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:37.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:37.299 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:38.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:38 smithi161 ceph-mon[84406]: pgmap v366: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 81 KiB/s rd, 85 B/s wr, 126 op/s 2024-06-01T02:13:38.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:38 smithi161 ceph-mon[84406]: Upgrade: Updating iscsi.foo.smithi116.gjuxzb 2024-06-01T02:13:38.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:38.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi116.gjuxzb", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-01T02:13:38.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:38.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:38 smithi161 ceph-mon[84406]: Deploying daemon iscsi.foo.smithi116.gjuxzb on smithi116 2024-06-01T02:13:39.042 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:38 smithi116 ceph-mon[100606]: pgmap v366: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 81 KiB/s rd, 85 B/s wr, 126 op/s 2024-06-01T02:13:39.042 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:38 smithi116 ceph-mon[100606]: Upgrade: Updating iscsi.foo.smithi116.gjuxzb 2024-06-01T02:13:39.042 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:39.042 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi116.gjuxzb", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-01T02:13:39.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:39.045 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:38 smithi116 ceph-mon[100606]: Deploying daemon iscsi.foo.smithi116.gjuxzb on smithi116 2024-06-01T02:13:39.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:38 smithi116 ceph-mon[89502]: pgmap v366: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 81 KiB/s rd, 85 B/s wr, 126 op/s 2024-06-01T02:13:39.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:38 smithi116 ceph-mon[89502]: Upgrade: Updating iscsi.foo.smithi116.gjuxzb 2024-06-01T02:13:39.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:39.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi116.gjuxzb", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-01T02:13:39.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:39.045 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:38 smithi116 ceph-mon[89502]: Deploying daemon iscsi.foo.smithi116.gjuxzb on smithi116 2024-06-01T02:13:41.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:40 smithi116 ceph-mon[89502]: pgmap v367: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 51 op/s 2024-06-01T02:13:41.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:40 smithi116 ceph-mon[100606]: pgmap v367: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 51 op/s 2024-06-01T02:13:41.193 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:40 smithi161 ceph-mon[84406]: pgmap v367: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 51 op/s 2024-06-01T02:13:43.068 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:42 smithi116 ceph-mon[100606]: pgmap v368: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 8.1 KiB/s rd, 0 B/s wr, 12 op/s 2024-06-01T02:13:43.069 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:42 smithi116 ceph-mon[89502]: pgmap v368: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 8.1 KiB/s rd, 0 B/s wr, 12 op/s 2024-06-01T02:13:43.193 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:42 smithi161 ceph-mon[84406]: pgmap v368: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 8.1 KiB/s rd, 0 B/s wr, 12 op/s 2024-06-01T02:13:44.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:43 smithi161 ceph-mon[84406]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:13:44.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:43 smithi116 ceph-mon[100606]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:13:44.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:43 smithi116 ceph-mon[89502]: from='client.25414 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:13:45.193 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:44 smithi161 ceph-mon[84406]: pgmap v369: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:13:45.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:44 smithi116 ceph-mon[89502]: pgmap v369: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:13:45.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:44 smithi116 ceph-mon[100606]: pgmap v369: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:13:47.193 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:13:46 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:13:46] "GET /metrics HTTP/1.1" 200 38699 "" "Prometheus/2.33.4" 2024-06-01T02:13:47.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:46 smithi161 ceph-mon[84406]: pgmap v370: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:13:47.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:46 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:13:47.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:46 smithi116 ceph-mon[100606]: pgmap v370: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:13:47.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:46 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:13:47.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:46 smithi116 ceph-mon[89502]: pgmap v370: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:13:47.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:46 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:13:49.193 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:48 smithi161 ceph-mon[84406]: pgmap v371: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:13:49.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:48 smithi116 ceph-mon[89502]: pgmap v371: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:13:49.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:48 smithi116 ceph-mon[100606]: pgmap v371: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:13:49.945 INFO:teuthology.orchestra.run.smithi116.stdout:true 2024-06-01T02:13:50.622 INFO:teuthology.orchestra.run.smithi116.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-01T02:13:50.622 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi116 smithi116 running 19s ago 22m - - 2024-06-01T02:13:50.623 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi161 smithi161 running 14s ago 22m - - 2024-06-01T02:13:50.623 INFO:teuthology.orchestra.run.smithi116.stdout:alertmanager.a smithi116 *:9093,9094 running (11m) 19s ago 16m 22.4M - 0.23.0 ba2b418f427c e0c297162ded 2024-06-01T02:13:50.623 INFO:teuthology.orchestra.run.smithi116.stdout:grafana.a smithi161 *:3000 running (11m) 14s ago 16m 54.7M - 8.3.5 dad864ee21e9 00b0a78ee88e 2024-06-01T02:13:50.623 INFO:teuthology.orchestra.run.smithi116.stdout:iscsi.foo.smithi116.gjuxzb smithi116 running (10m) 19s ago 15m 62.9M - 3.5 e1d6a67b021e 0e191fe3e3bb 2024-06-01T02:13:50.623 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.x smithi161 *:8443,9283,8765 running (10m) 14s ago 21m 542M - 19.0.0-2946-g3d89accf b045ac545f82 2d7fbb85e5f7 2024-06-01T02:13:50.623 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.y smithi116 *:8443,9283,8765 running (9m) 19s ago 23m 456M - 19.0.0-2946-g3d89accf b045ac545f82 bc9627c27a57 2024-06-01T02:13:50.623 INFO:teuthology.orchestra.run.smithi116.stdout:mon.a smithi116 running (9m) 19s ago 23m 80.2M 2048M 19.0.0-2946-g3d89accf b045ac545f82 b5e9b6f873ff 2024-06-01T02:13:50.623 INFO:teuthology.orchestra.run.smithi116.stdout:mon.b smithi161 running (7m) 14s ago 21m 56.7M 2048M 19.0.0-2946-g3d89accf b045ac545f82 aaa08c03cfc8 2024-06-01T02:13:50.623 INFO:teuthology.orchestra.run.smithi116.stdout:mon.c smithi116 running (8m) 19s ago 22m 60.3M 2048M 19.0.0-2946-g3d89accf b045ac545f82 bd0fb0e603fa 2024-06-01T02:13:50.623 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.a smithi116 *:9100 running (11m) 19s ago 17m 17.2M - 1.3.1 1dbe0e931976 e347748d17bb 2024-06-01T02:13:50.623 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.b smithi161 *:9100 running (11m) 14s ago 17m 17.9M - 1.3.1 1dbe0e931976 6e66182f5a1a 2024-06-01T02:13:50.623 INFO:teuthology.orchestra.run.smithi116.stdout:osd.0 smithi116 running (6m) 19s ago 21m 61.5M 4096M 19.0.0-2946-g3d89accf b045ac545f82 b4313d121b9d 2024-06-01T02:13:50.623 INFO:teuthology.orchestra.run.smithi116.stdout:osd.1 smithi116 running (6m) 19s ago 20m 65.1M 4096M 19.0.0-2946-g3d89accf b045ac545f82 dccdbeac5aae 2024-06-01T02:13:50.623 INFO:teuthology.orchestra.run.smithi116.stdout:osd.2 smithi116 running (5m) 19s ago 20m 56.9M 4096M 19.0.0-2946-g3d89accf b045ac545f82 1ad95efda404 2024-06-01T02:13:50.624 INFO:teuthology.orchestra.run.smithi116.stdout:osd.3 smithi116 running (5m) 19s ago 20m 64.2M 4096M 19.0.0-2946-g3d89accf b045ac545f82 bd00ad2c2556 2024-06-01T02:13:50.624 INFO:teuthology.orchestra.run.smithi116.stdout:osd.4 smithi161 running (4m) 14s ago 19m 64.9M 4096M 19.0.0-2946-g3d89accf b045ac545f82 0084b9edf7a0 2024-06-01T02:13:50.624 INFO:teuthology.orchestra.run.smithi116.stdout:osd.5 smithi161 running (3m) 14s ago 19m 57.6M 4096M 19.0.0-2946-g3d89accf b045ac545f82 b3849cdeb707 2024-06-01T02:13:50.624 INFO:teuthology.orchestra.run.smithi116.stdout:osd.6 smithi161 running (2m) 14s ago 18m 52.3M 4096M 19.0.0-2946-g3d89accf b045ac545f82 6b87d9299c8d 2024-06-01T02:13:50.624 INFO:teuthology.orchestra.run.smithi116.stdout:osd.7 smithi161 running (86s) 14s ago 18m 59.6M 4096M 19.0.0-2946-g3d89accf b045ac545f82 400bb59c4249 2024-06-01T02:13:50.624 INFO:teuthology.orchestra.run.smithi116.stdout:prometheus.a smithi161 *:9095 running (10m) 14s ago 17m 79.1M - 2.33.4 514e6a882f6e e977f6ca1d73 2024-06-01T02:13:50.624 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi116.vzjrxj smithi116 *:8000 running (77s) 19s ago 15m 78.3M - 19.0.0-2946-g3d89accf b045ac545f82 376c2daccbd5 2024-06-01T02:13:50.624 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi161.ashqxz smithi161 *:8000 running (35s) 14s ago 15m 77.7M - 19.0.0-2946-g3d89accf b045ac545f82 6a0c3a3546be 2024-06-01T02:13:50.624 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi116.hokbmf smithi116 *:80 running (71s) 19s ago 15m 78.2M - 19.0.0-2946-g3d89accf b045ac545f82 3b0ab9880bcf 2024-06-01T02:13:50.624 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi161.axjlzw smithi161 *:80 running (25s) 14s ago 15m 75.5M - 19.0.0-2946-g3d89accf b045ac545f82 27533bf2d313 2024-06-01T02:13:51.051 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:13:51.051 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": { 2024-06-01T02:13:51.051 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 3 2024-06-01T02:13:51.051 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:13:51.051 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": { 2024-06-01T02:13:51.051 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-01T02:13:51.051 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:13:51.052 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": { 2024-06-01T02:13:51.052 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 8 2024-06-01T02:13:51.052 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:13:51.052 INFO:teuthology.orchestra.run.smithi116.stdout: "rgw": { 2024-06-01T02:13:51.052 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 5 2024-06-01T02:13:51.052 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:13:51.052 INFO:teuthology.orchestra.run.smithi116.stdout: "overall": { 2024-06-01T02:13:51.052 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 18 2024-06-01T02:13:51.052 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T02:13:51.052 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:13:51.134 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:50 smithi116 ceph-mon[100606]: pgmap v372: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:13:51.134 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:50 smithi116 ceph-mon[100606]: from='client.64897 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:13:51.134 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:50 smithi116 ceph-mon[89502]: pgmap v372: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:13:51.134 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:50 smithi116 ceph-mon[89502]: from='client.64897 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:13:51.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:50 smithi161 ceph-mon[84406]: pgmap v372: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:13:51.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:50 smithi161 ceph-mon[84406]: from='client.64897 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:13:51.404 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:13:51.404 INFO:teuthology.orchestra.run.smithi116.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-01T02:13:51.405 INFO:teuthology.orchestra.run.smithi116.stdout: "in_progress": true, 2024-06-01T02:13:51.405 INFO:teuthology.orchestra.run.smithi116.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-01T02:13:51.405 INFO:teuthology.orchestra.run.smithi116.stdout: "services_complete": [ 2024-06-01T02:13:51.405 INFO:teuthology.orchestra.run.smithi116.stdout: "mon", 2024-06-01T02:13:51.405 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr", 2024-06-01T02:13:51.405 INFO:teuthology.orchestra.run.smithi116.stdout: "osd", 2024-06-01T02:13:51.405 INFO:teuthology.orchestra.run.smithi116.stdout: "rgw" 2024-06-01T02:13:51.405 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-06-01T02:13:51.405 INFO:teuthology.orchestra.run.smithi116.stdout: "progress": "17/23 daemons upgraded", 2024-06-01T02:13:51.405 INFO:teuthology.orchestra.run.smithi116.stdout: "message": "Currently upgrading iscsi daemons", 2024-06-01T02:13:51.405 INFO:teuthology.orchestra.run.smithi116.stdout: "is_paused": false 2024-06-01T02:13:51.405 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:13:51.889 INFO:teuthology.orchestra.run.smithi116.stdout:HEALTH_OK 2024-06-01T02:13:52.037 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:51 smithi116 ceph-mon[89502]: from='client.64903 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:13:52.037 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:51 smithi116 ceph-mon[89502]: from='client.64909 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:13:52.037 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:51 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/2433035778' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:52.037 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:52.038 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:52.038 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:51 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:52.038 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:51 smithi116 ceph-mon[100606]: from='client.64903 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:13:52.038 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:51 smithi116 ceph-mon[100606]: from='client.64909 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:13:52.039 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:51 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/2433035778' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:52.039 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:52.039 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:52.039 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:51 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:52.144 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:51 smithi161 ceph-mon[84406]: from='client.64903 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:13:52.144 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:51 smithi161 ceph-mon[84406]: from='client.64909 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:13:52.144 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:51 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/2433035778' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:13:52.145 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:52.145 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:52.145 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:51 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:52.443 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:13:52 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:13:52.147+0000 7fcea3e68700 -1 log_channel(cephadm) log [ERR] : Upgrade: Paused due to UPGRADE_REDEPLOY_DAEMON: Upgrading daemon iscsi.foo.smithi116.gjuxzb on host smithi116 failed. 2024-06-01T02:13:53.015 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:52 smithi116 ceph-mon[100606]: from='client.64921 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:13:53.017 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:52 smithi116 ceph-mon[100606]: pgmap v373: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:13:53.017 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:52 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/15620960' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:13:53.017 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:53.018 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:53.018 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:53.018 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:53.018 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:53.018 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:53.018 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:52 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:53.019 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:52 smithi116 ceph-mon[89502]: from='client.64921 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:13:53.019 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:52 smithi116 ceph-mon[89502]: pgmap v373: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:13:53.019 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:52 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/15620960' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:13:53.019 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:53.020 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:53.020 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:53.020 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:53.020 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:53.020 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:53.020 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:52 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:53.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:52 smithi161 ceph-mon[84406]: from='client.64921 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:13:53.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:52 smithi161 ceph-mon[84406]: pgmap v373: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:13:53.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:52 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/15620960' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:13:53.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:53.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:53.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:53.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:53.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:53.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:53.195 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:52 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:54.095 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:53 smithi116 ceph-mon[100606]: Upgrade: Paused due to UPGRADE_REDEPLOY_DAEMON: Upgrading daemon iscsi.foo.smithi116.gjuxzb on host smithi116 failed. 2024-06-01T02:13:54.095 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:53 smithi116 ceph-mon[100606]: pgmap v374: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 969 B/s rd, 0 op/s 2024-06-01T02:13:54.095 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:53 smithi116 ceph-mon[100606]: Health check failed: Upgrading daemon iscsi.foo.smithi116.gjuxzb on host smithi116 failed. (UPGRADE_REDEPLOY_DAEMON) 2024-06-01T02:13:54.095 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:53 smithi116 ceph-mon[89502]: Upgrade: Paused due to UPGRADE_REDEPLOY_DAEMON: Upgrading daemon iscsi.foo.smithi116.gjuxzb on host smithi116 failed. 2024-06-01T02:13:54.095 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:53 smithi116 ceph-mon[89502]: pgmap v374: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 969 B/s rd, 0 op/s 2024-06-01T02:13:54.095 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:53 smithi116 ceph-mon[89502]: Health check failed: Upgrading daemon iscsi.foo.smithi116.gjuxzb on host smithi116 failed. (UPGRADE_REDEPLOY_DAEMON) 2024-06-01T02:13:54.193 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:53 smithi161 ceph-mon[84406]: Upgrade: Paused due to UPGRADE_REDEPLOY_DAEMON: Upgrading daemon iscsi.foo.smithi116.gjuxzb on host smithi116 failed. 2024-06-01T02:13:54.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:53 smithi161 ceph-mon[84406]: pgmap v374: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 969 B/s rd, 0 op/s 2024-06-01T02:13:54.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:53 smithi161 ceph-mon[84406]: Health check failed: Upgrading daemon iscsi.foo.smithi116.gjuxzb on host smithi116 failed. (UPGRADE_REDEPLOY_DAEMON) 2024-06-01T02:13:56.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:55 smithi161 ceph-mon[84406]: pgmap v375: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 775 B/s rd, 0 op/s 2024-06-01T02:13:56.266 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:55 smithi116 ceph-mon[89502]: pgmap v375: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 775 B/s rd, 0 op/s 2024-06-01T02:13:56.266 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:55 smithi116 ceph-mon[100606]: pgmap v375: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 775 B/s rd, 0 op/s 2024-06-01T02:13:57.090 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:13:56 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:13:56] "GET /metrics HTTP/1.1" 200 38699 "" "Prometheus/2.33.4" 2024-06-01T02:13:57.361 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:57 smithi161 ceph-mon[84406]: pgmap v376: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 775 B/s rd, 0 op/s 2024-06-01T02:13:57.361 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:57 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:57.361 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:57 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:57.361 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:57 smithi161 ceph-mon[84406]: pgmap v377: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 354 B/s rd, 0 op/s 2024-06-01T02:13:57.362 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:57 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:57.362 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:57 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:57.362 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:57 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:57.362 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:57 smithi161 ceph-mon[84406]: pgmap v378: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 459 B/s rd, 0 op/s 2024-06-01T02:13:57.362 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:57 smithi161 ceph-mon[84406]: pgmap v379: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 656 B/s rd, 0 op/s 2024-06-01T02:13:57.362 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:57 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:57.362 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:57 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:57.362 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:57 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:57.362 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:57 smithi161 ceph-mon[84406]: pgmap v380: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 712 B/s rd, 0 op/s 2024-06-01T02:13:57.362 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:57 smithi161 ceph-mon[84406]: pgmap v381: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:13:57.362 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:57 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:57.362 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:57 smithi161 ceph-mon[84406]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-06-01T02:13:57.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:57 smithi116 ceph-mon[89502]: pgmap v376: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 775 B/s rd, 0 op/s 2024-06-01T02:13:57.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:57.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:57.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:57 smithi116 ceph-mon[89502]: pgmap v377: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 354 B/s rd, 0 op/s 2024-06-01T02:13:57.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:57.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:57.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:57.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:57 smithi116 ceph-mon[89502]: pgmap v378: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 459 B/s rd, 0 op/s 2024-06-01T02:13:57.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:57 smithi116 ceph-mon[89502]: pgmap v379: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 656 B/s rd, 0 op/s 2024-06-01T02:13:57.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:57.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:57.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:57.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:57 smithi116 ceph-mon[89502]: pgmap v380: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 712 B/s rd, 0 op/s 2024-06-01T02:13:57.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:57 smithi116 ceph-mon[89502]: pgmap v381: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:13:57.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:57.544 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:57 smithi116 ceph-mon[89502]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-06-01T02:13:57.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:57 smithi116 ceph-mon[100606]: pgmap v376: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 775 B/s rd, 0 op/s 2024-06-01T02:13:57.544 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:57 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:57.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:57 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:57.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:57 smithi116 ceph-mon[100606]: pgmap v377: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 354 B/s rd, 0 op/s 2024-06-01T02:13:57.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:57 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:57.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:57 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:13:57.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:57 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:13:57.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:57 smithi116 ceph-mon[100606]: pgmap v378: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 459 B/s rd, 0 op/s 2024-06-01T02:13:57.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:57 smithi116 ceph-mon[100606]: pgmap v379: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 656 B/s rd, 0 op/s 2024-06-01T02:13:57.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:57 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:57.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:57 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:57.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:57 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:57.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:57 smithi116 ceph-mon[100606]: pgmap v380: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 712 B/s rd, 0 op/s 2024-06-01T02:13:57.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:57 smithi116 ceph-mon[100606]: pgmap v381: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:13:57.545 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:57 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:13:57.546 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:57 smithi116 ceph-mon[100606]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-06-01T02:13:59.754 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:13:59 smithi161 ceph-mon[84406]: pgmap v382: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:13:59.792 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:13:59 smithi116 ceph-mon[89502]: pgmap v382: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:13:59.792 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:13:59 smithi116 ceph-mon[100606]: pgmap v382: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:14:02.193 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:01 smithi161 ceph-mon[84406]: pgmap v383: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:14:02.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:01 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:02.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:01 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:14:02.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:01 smithi116 ceph-mon[89502]: pgmap v383: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:14:02.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:02.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:01 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:14:02.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:01 smithi116 ceph-mon[100606]: pgmap v383: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:14:02.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:01 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:02.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:01 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:14:04.193 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:03 smithi161 ceph-mon[84406]: pgmap v384: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:14:04.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:03 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/1110331165' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-01T02:14:04.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:03 smithi116 ceph-mon[89502]: pgmap v384: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:14:04.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:03 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/1110331165' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-01T02:14:04.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:03 smithi116 ceph-mon[100606]: pgmap v384: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:14:04.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:03 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/1110331165' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-01T02:14:06.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:05 smithi161 ceph-mon[84406]: pgmap v385: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:14:06.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:05 smithi116 ceph-mon[89502]: pgmap v385: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:14:06.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:05 smithi116 ceph-mon[100606]: pgmap v385: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:14:07.193 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:14:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:14:06] "GET /metrics HTTP/1.1" 200 38775 "" "Prometheus/2.33.4" 2024-06-01T02:14:08.193 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:07 smithi161 ceph-mon[84406]: pgmap v386: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:14:08.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:07 smithi116 ceph-mon[89502]: pgmap v386: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:14:08.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:07 smithi116 ceph-mon[100606]: pgmap v386: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail 2024-06-01T02:14:10.193 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:09 smithi161 ceph-mon[84406]: pgmap v387: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 426 B/s rd, 0 op/s 2024-06-01T02:14:10.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:09 smithi116 ceph-mon[89502]: pgmap v387: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 426 B/s rd, 0 op/s 2024-06-01T02:14:10.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:09 smithi116 ceph-mon[100606]: pgmap v387: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 426 B/s rd, 0 op/s 2024-06-01T02:14:12.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:11 smithi116 ceph-mon[89502]: pgmap v388: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 426 B/s rd, 0 op/s 2024-06-01T02:14:12.042 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:11 smithi116 ceph-mon[100606]: pgmap v388: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 426 B/s rd, 0 op/s 2024-06-01T02:14:12.193 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:11 smithi161 ceph-mon[84406]: pgmap v388: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 426 B/s rd, 0 op/s 2024-06-01T02:14:14.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:13 smithi161 ceph-mon[84406]: pgmap v389: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:14:14.252 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:13 smithi116 ceph-mon[89502]: pgmap v389: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:14:14.252 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:13 smithi116 ceph-mon[100606]: pgmap v389: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:14:15.042 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:14 smithi116 ceph-mon[100606]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:14:15.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:14 smithi116 ceph-mon[89502]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:14:15.193 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:14 smithi161 ceph-mon[84406]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:14:16.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:15 smithi161 ceph-mon[84406]: pgmap v390: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-01T02:14:16.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:15 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:14:16.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:15 smithi116 ceph-mon[100606]: pgmap v390: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-01T02:14:16.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:15 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:14:16.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:15 smithi116 ceph-mon[89502]: pgmap v390: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-01T02:14:16.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:15 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:14:16.881 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:14:16 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:14:16] "GET /metrics HTTP/1.1" 200 38773 "" "Prometheus/2.33.4" 2024-06-01T02:14:17.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:17.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:17.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:17 smithi161 ceph-mon[84406]: pgmap v391: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 857 B/s rd, 0 op/s 2024-06-01T02:14:17.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:17 smithi161 ceph-mon[84406]: pgmap v392: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-06-01T02:14:17.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:17.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:17.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:17.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:17 smithi161 ceph-mon[84406]: pgmap v393: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 630 B/s rd, 0 op/s 2024-06-01T02:14:17.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:17 smithi161 ceph-mon[84406]: pgmap v394: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 835 B/s rd, 0 op/s 2024-06-01T02:14:17.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:17.792 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:17.792 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:17.792 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:17 smithi116 ceph-mon[100606]: pgmap v391: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 857 B/s rd, 0 op/s 2024-06-01T02:14:17.792 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:17 smithi116 ceph-mon[100606]: pgmap v392: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-06-01T02:14:17.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:17.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:17.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:17.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:17 smithi116 ceph-mon[100606]: pgmap v393: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 630 B/s rd, 0 op/s 2024-06-01T02:14:17.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:17 smithi116 ceph-mon[100606]: pgmap v394: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 835 B/s rd, 0 op/s 2024-06-01T02:14:17.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:17.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:17.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:17.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:17 smithi116 ceph-mon[89502]: pgmap v391: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 857 B/s rd, 0 op/s 2024-06-01T02:14:17.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:17 smithi116 ceph-mon[89502]: pgmap v392: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-06-01T02:14:17.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:17.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:17.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:17.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:17 smithi116 ceph-mon[89502]: pgmap v393: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 630 B/s rd, 0 op/s 2024-06-01T02:14:17.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:17 smithi116 ceph-mon[89502]: pgmap v394: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 835 B/s rd, 0 op/s 2024-06-01T02:14:17.794 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:19.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:19.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:19.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:18 smithi161 ceph-mon[84406]: pgmap v395: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 946 B/s rd, 0 op/s 2024-06-01T02:14:19.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:19.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:14:19.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:14:19.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:19.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:19.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:19.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:18 smithi116 ceph-mon[100606]: pgmap v395: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 946 B/s rd, 0 op/s 2024-06-01T02:14:19.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:19.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:14:19.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:14:19.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:19.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:19.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:19.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:18 smithi116 ceph-mon[89502]: pgmap v395: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 946 B/s rd, 0 op/s 2024-06-01T02:14:19.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:19.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:14:19.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:14:19.294 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:20.193 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:19 smithi161 ceph-mon[84406]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-06-01T02:14:20.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:19 smithi116 ceph-mon[89502]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-06-01T02:14:20.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:19 smithi116 ceph-mon[100606]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-06-01T02:14:21.193 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:20 smithi161 ceph-mon[84406]: pgmap v396: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 946 B/s rd, 0 op/s 2024-06-01T02:14:21.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:20 smithi116 ceph-mon[100606]: pgmap v396: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 946 B/s rd, 0 op/s 2024-06-01T02:14:21.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:20 smithi116 ceph-mon[89502]: pgmap v396: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 946 B/s rd, 0 op/s 2024-06-01T02:14:22.259 INFO:teuthology.orchestra.run.smithi116.stdout:true 2024-06-01T02:14:22.588 INFO:teuthology.orchestra.run.smithi116.stdout:"Error: UPGRADE_REDEPLOY_DAEMON: Upgrading daemon iscsi.foo.smithi116.gjuxzb on host smithi116 failed." 2024-06-01T02:14:23.193 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:22 smithi161 ceph-mon[84406]: pgmap v397: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 935 B/s rd, 0 op/s 2024-06-01T02:14:23.276 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:22 smithi116 ceph-mon[89502]: pgmap v397: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 935 B/s rd, 0 op/s 2024-06-01T02:14:23.276 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:22 smithi116 ceph-mon[100606]: pgmap v397: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 935 B/s rd, 0 op/s 2024-06-01T02:14:23.299 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph orch ps' 2024-06-01T02:14:24.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:23 smithi161 ceph-mon[84406]: from='client.44976 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:14:24.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:23 smithi161 ceph-mon[84406]: from='client.44982 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:14:24.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:23 smithi116 ceph-mon[100606]: from='client.44976 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:14:24.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:23 smithi116 ceph-mon[100606]: from='client.44982 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:14:24.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:23 smithi116 ceph-mon[89502]: from='client.44976 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:14:24.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:23 smithi116 ceph-mon[89502]: from='client.44982 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:14:25.193 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:24 smithi161 ceph-mon[84406]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:14:25.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:24 smithi161 ceph-mon[84406]: pgmap v398: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-01T02:14:25.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:24 smithi116 ceph-mon[89502]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:14:25.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:24 smithi116 ceph-mon[89502]: pgmap v398: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-01T02:14:25.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:24 smithi116 ceph-mon[100606]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:14:25.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:24 smithi116 ceph-mon[100606]: pgmap v398: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-01T02:14:25.359 INFO:teuthology.orchestra.run.smithi116.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-01T02:14:25.359 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi116 smithi116 running 7s ago 23m - - 2024-06-01T02:14:25.359 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi161 smithi161 running 8s ago 22m - - 2024-06-01T02:14:25.359 INFO:teuthology.orchestra.run.smithi116.stdout:alertmanager.a smithi116 *:9093,9094 running (12m) 7s ago 17m - - 0.23.0 ba2b418f427c e0c297162ded 2024-06-01T02:14:25.359 INFO:teuthology.orchestra.run.smithi116.stdout:grafana.a smithi161 *:3000 running (12m) 8s ago 17m 55.1M - 8.3.5 dad864ee21e9 00b0a78ee88e 2024-06-01T02:14:25.359 INFO:teuthology.orchestra.run.smithi116.stdout:iscsi.foo.smithi116.gjuxzb smithi116 running (22s) 7s ago 16m - - 3.8 b045ac545f82 fe8075e0acba 2024-06-01T02:14:25.359 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.x smithi161 *:8443,9283,8765 running (11m) 8s ago 22m 542M - 19.0.0-2946-g3d89accf b045ac545f82 2d7fbb85e5f7 2024-06-01T02:14:25.359 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.y smithi116 *:8443,9283,8765 running (10m) 7s ago 23m - - 19.0.0-2946-g3d89accf b045ac545f82 bc9627c27a57 2024-06-01T02:14:25.359 INFO:teuthology.orchestra.run.smithi116.stdout:mon.a smithi116 running (9m) 7s ago 24m - 2048M 19.0.0-2946-g3d89accf b045ac545f82 b5e9b6f873ff 2024-06-01T02:14:25.359 INFO:teuthology.orchestra.run.smithi116.stdout:mon.b smithi161 running (8m) 8s ago 22m 56.7M 2048M 19.0.0-2946-g3d89accf b045ac545f82 aaa08c03cfc8 2024-06-01T02:14:25.359 INFO:teuthology.orchestra.run.smithi116.stdout:mon.c smithi116 running (8m) 7s ago 22m - 2048M 19.0.0-2946-g3d89accf b045ac545f82 bd0fb0e603fa 2024-06-01T02:14:25.360 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.a smithi116 *:9100 running (12m) 7s ago 18m - - 1.3.1 1dbe0e931976 e347748d17bb 2024-06-01T02:14:25.360 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.b smithi161 *:9100 running (12m) 8s ago 18m 17.9M - 1.3.1 1dbe0e931976 6e66182f5a1a 2024-06-01T02:14:25.360 INFO:teuthology.orchestra.run.smithi116.stdout:osd.0 smithi116 running (7m) 7s ago 21m - 4096M 19.0.0-2946-g3d89accf b045ac545f82 b4313d121b9d 2024-06-01T02:14:25.360 INFO:teuthology.orchestra.run.smithi116.stdout:osd.1 smithi116 running (6m) 7s ago 21m - 4096M 19.0.0-2946-g3d89accf b045ac545f82 dccdbeac5aae 2024-06-01T02:14:25.360 INFO:teuthology.orchestra.run.smithi116.stdout:osd.2 smithi116 running (6m) 7s ago 21m - 4096M 19.0.0-2946-g3d89accf b045ac545f82 1ad95efda404 2024-06-01T02:14:25.360 INFO:teuthology.orchestra.run.smithi116.stdout:osd.3 smithi116 running (5m) 7s ago 20m - 4096M 19.0.0-2946-g3d89accf b045ac545f82 bd00ad2c2556 2024-06-01T02:14:25.360 INFO:teuthology.orchestra.run.smithi116.stdout:osd.4 smithi161 running (4m) 8s ago 20m 65.3M 4096M 19.0.0-2946-g3d89accf b045ac545f82 0084b9edf7a0 2024-06-01T02:14:25.360 INFO:teuthology.orchestra.run.smithi116.stdout:osd.5 smithi161 running (3m) 8s ago 19m 58.2M 4096M 19.0.0-2946-g3d89accf b045ac545f82 b3849cdeb707 2024-06-01T02:14:25.360 INFO:teuthology.orchestra.run.smithi116.stdout:osd.6 smithi161 running (3m) 8s ago 19m 52.5M 4096M 19.0.0-2946-g3d89accf b045ac545f82 6b87d9299c8d 2024-06-01T02:14:25.360 INFO:teuthology.orchestra.run.smithi116.stdout:osd.7 smithi161 running (2m) 8s ago 19m 59.6M 4096M 19.0.0-2946-g3d89accf b045ac545f82 400bb59c4249 2024-06-01T02:14:25.360 INFO:teuthology.orchestra.run.smithi116.stdout:prometheus.a smithi161 *:9095 running (11m) 8s ago 18m 79.1M - 2.33.4 514e6a882f6e e977f6ca1d73 2024-06-01T02:14:25.360 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi116.vzjrxj smithi116 *:8000 running (111s) 7s ago 16m - - 19.0.0-2946-g3d89accf b045ac545f82 376c2daccbd5 2024-06-01T02:14:25.360 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi161.ashqxz smithi161 *:8000 running (70s) 8s ago 16m 78.1M - 19.0.0-2946-g3d89accf b045ac545f82 6a0c3a3546be 2024-06-01T02:14:25.360 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi116.hokbmf smithi116 *:80 running (105s) 7s ago 16m - - 19.0.0-2946-g3d89accf b045ac545f82 3b0ab9880bcf 2024-06-01T02:14:25.361 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi161.axjlzw smithi161 *:80 running (60s) 8s ago 16m 75.9M - 19.0.0-2946-g3d89accf b045ac545f82 27533bf2d313 2024-06-01T02:14:26.065 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph versions' 2024-06-01T02:14:27.193 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:14:26 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:14:26] "GET /metrics HTTP/1.1" 200 38773 "" "Prometheus/2.33.4" 2024-06-01T02:14:27.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:26 smithi161 ceph-mon[84406]: from='client.64969 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:14:27.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:26 smithi161 ceph-mon[84406]: pgmap v399: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-01T02:14:27.245 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:26 smithi116 ceph-mon[100606]: from='client.64969 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:14:27.245 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:26 smithi116 ceph-mon[100606]: pgmap v399: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-01T02:14:27.245 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:26 smithi116 ceph-mon[89502]: from='client.64969 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:14:27.245 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:26 smithi116 ceph-mon[89502]: pgmap v399: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-01T02:14:29.119 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:28 smithi116 ceph-mon[89502]: pgmap v400: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-01T02:14:29.120 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:28 smithi116 ceph-mon[100606]: pgmap v400: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-01T02:14:29.193 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:28 smithi161 ceph-mon[84406]: pgmap v400: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-01T02:14:29.526 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:14:29.526 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": { 2024-06-01T02:14:29.527 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 3 2024-06-01T02:14:29.527 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:14:29.527 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": { 2024-06-01T02:14:29.527 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-01T02:14:29.527 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:14:29.527 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": { 2024-06-01T02:14:29.527 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 8 2024-06-01T02:14:29.527 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:14:29.527 INFO:teuthology.orchestra.run.smithi116.stdout: "rgw": { 2024-06-01T02:14:29.527 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 4 2024-06-01T02:14:29.528 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:14:29.528 INFO:teuthology.orchestra.run.smithi116.stdout: "overall": { 2024-06-01T02:14:29.528 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 17 2024-06-01T02:14:29.528 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T02:14:29.528 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:14:30.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:29 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/1112234382' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:14:30.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:29 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/1112234382' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:14:30.424 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-06-01T02:14:30.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:29 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/1112234382' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:14:31.176 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:30 smithi116 ceph-mon[100606]: pgmap v401: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:14:31.177 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:31.177 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:30 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:14:31.177 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:30 smithi116 ceph-mon[89502]: pgmap v401: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:14:31.177 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:31.177 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:30 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:14:31.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:30 smithi161 ceph-mon[84406]: pgmap v401: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:14:31.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:31.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:30 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:14:32.496 INFO:teuthology.orchestra.run.smithi116.stdout:wait for servicemap items w/ changing names to refresh 2024-06-01T02:14:33.120 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:32 smithi116 ceph-mon[89502]: pgmap v402: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:14:33.120 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:32 smithi116 ceph-mon[100606]: pgmap v402: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:14:33.165 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'sleep 60' 2024-06-01T02:14:33.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:32 smithi161 ceph-mon[84406]: pgmap v402: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:14:35.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:34 smithi116 ceph-mon[89502]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:14:35.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:34 smithi116 ceph-mon[89502]: pgmap v403: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:14:35.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:34 smithi116 ceph-mon[100606]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:14:35.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:34 smithi116 ceph-mon[100606]: pgmap v403: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:14:35.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:34 smithi161 ceph-mon[84406]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:14:35.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:34 smithi161 ceph-mon[84406]: pgmap v403: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:14:36.693 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:36.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:36.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:36.694 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:36 smithi161 ceph-mon[84406]: pgmap v404: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-01T02:14:36.792 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:36.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:36.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:36.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:36 smithi116 ceph-mon[89502]: pgmap v404: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-01T02:14:36.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:36.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:36.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:36.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:36 smithi116 ceph-mon[100606]: pgmap v404: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-01T02:14:37.193 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:14:36 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:14:36] "GET /metrics HTTP/1.1" 200 38775 "" "Prometheus/2.33.4" 2024-06-01T02:14:38.023 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:38.023 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:38.023 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:37 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:38.042 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:38.042 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:38.042 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:37 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:38.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:38.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:38.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:37 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:38.991 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:38 smithi161 ceph-mon[84406]: pgmap v405: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:14:39.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:38 smithi116 ceph-mon[89502]: pgmap v405: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:14:39.042 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:38 smithi116 ceph-mon[100606]: pgmap v405: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:14:41.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:40 smithi161 ceph-mon[84406]: pgmap v406: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:14:41.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:40 smithi116 ceph-mon[89502]: pgmap v406: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:14:41.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:40 smithi116 ceph-mon[100606]: pgmap v406: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:14:43.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:42 smithi161 ceph-mon[84406]: pgmap v407: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:14:43.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:42 smithi116 ceph-mon[89502]: pgmap v407: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:14:43.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:42 smithi116 ceph-mon[100606]: pgmap v407: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:14:45.193 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:44 smithi161 ceph-mon[84406]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:14:45.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:44 smithi161 ceph-mon[84406]: pgmap v408: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:14:45.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:44 smithi116 ceph-mon[89502]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:14:45.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:44 smithi116 ceph-mon[89502]: pgmap v408: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:14:45.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:44 smithi116 ceph-mon[100606]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:14:45.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:44 smithi116 ceph-mon[100606]: pgmap v408: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:14:46.222 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:14:46.223 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:14:46.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:14:47.193 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:46 smithi161 ceph-mon[84406]: pgmap v409: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:14:47.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:14:46 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:14:46] "GET /metrics HTTP/1.1" 200 38773 "" "Prometheus/2.33.4" 2024-06-01T02:14:47.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:46 smithi116 ceph-mon[89502]: pgmap v409: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:14:47.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:46 smithi116 ceph-mon[100606]: pgmap v409: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:14:49.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:48 smithi116 ceph-mon[100606]: pgmap v410: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:14:49.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:48 smithi116 ceph-mon[89502]: pgmap v410: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:14:49.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:48 smithi161 ceph-mon[84406]: pgmap v410: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:14:51.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:50 smithi116 ceph-mon[100606]: pgmap v411: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:14:51.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:50 smithi116 ceph-mon[89502]: pgmap v411: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:14:51.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:50 smithi161 ceph-mon[84406]: pgmap v411: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:14:53.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:52 smithi116 ceph-mon[100606]: pgmap v412: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:14:53.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:52 smithi116 ceph-mon[89502]: pgmap v412: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:14:53.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:52 smithi161 ceph-mon[84406]: pgmap v412: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:14:55.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:54 smithi116 ceph-mon[89502]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:14:55.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:54 smithi116 ceph-mon[89502]: pgmap v413: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:14:55.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:54 smithi116 ceph-mon[100606]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:14:55.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:54 smithi116 ceph-mon[100606]: pgmap v413: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:14:55.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:54 smithi161 ceph-mon[84406]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:14:55.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:54 smithi161 ceph-mon[84406]: pgmap v413: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:14:56.792 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:56.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:56.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:56.793 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:56 smithi116 ceph-mon[89502]: pgmap v414: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:14:56.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:56.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:56.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:56.793 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:56 smithi116 ceph-mon[100606]: pgmap v414: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:14:56.797 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:56.797 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:56.797 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:56.797 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:56 smithi161 ceph-mon[84406]: pgmap v414: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:14:57.193 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:14:56 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:14:56] "GET /metrics HTTP/1.1" 200 38773 "" "Prometheus/2.33.4" 2024-06-01T02:14:58.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:57 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:58.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:57 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:58.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:57 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:58.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:58.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:58.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:58.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:57 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:58.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:57 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:58.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:57 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:14:59.144 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:14:58 smithi161 ceph-mon[84406]: pgmap v415: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:14:59.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:14:58 smithi116 ceph-mon[100606]: pgmap v415: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:14:59.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:14:58 smithi116 ceph-mon[89502]: pgmap v415: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:01.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:00 smithi116 ceph-mon[100606]: pgmap v416: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:01.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:00 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:15:01.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:00 smithi116 ceph-mon[89502]: pgmap v416: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:01.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:15:01.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:00 smithi161 ceph-mon[84406]: pgmap v416: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:01.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:00 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:15:03.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:02 smithi116 ceph-mon[89502]: pgmap v417: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:03.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:02 smithi116 ceph-mon[100606]: pgmap v417: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:03.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:02 smithi161 ceph-mon[84406]: pgmap v417: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:05.237 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:04 smithi116 ceph-mon[100606]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:15:05.238 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:04 smithi116 ceph-mon[100606]: pgmap v418: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:05.238 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:04 smithi116 ceph-mon[89502]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:15:05.238 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:04 smithi116 ceph-mon[89502]: pgmap v418: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:05.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:04 smithi161 ceph-mon[84406]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:15:05.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:04 smithi161 ceph-mon[84406]: pgmap v418: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:07.193 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:06 smithi161 ceph-mon[84406]: pgmap v419: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:07.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:15:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:15:06] "GET /metrics HTTP/1.1" 200 38766 "" "Prometheus/2.33.4" 2024-06-01T02:15:07.228 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:06 smithi116 ceph-mon[100606]: pgmap v419: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:07.228 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:06 smithi116 ceph-mon[89502]: pgmap v419: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:09.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:08 smithi116 ceph-mon[89502]: pgmap v420: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:09.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:08 smithi116 ceph-mon[100606]: pgmap v420: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:09.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:08 smithi161 ceph-mon[84406]: pgmap v420: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:11.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:10 smithi116 ceph-mon[100606]: pgmap v421: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:11.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:10 smithi116 ceph-mon[89502]: pgmap v421: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:11.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:10 smithi161 ceph-mon[84406]: pgmap v421: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:13.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:12 smithi116 ceph-mon[89502]: pgmap v422: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:13.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:12 smithi116 ceph-mon[100606]: pgmap v422: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:13.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:12 smithi161 ceph-mon[84406]: pgmap v422: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:15.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:14 smithi116 ceph-mon[100606]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:15:15.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:14 smithi116 ceph-mon[100606]: pgmap v423: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:15.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:14 smithi116 ceph-mon[89502]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:15:15.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:14 smithi116 ceph-mon[89502]: pgmap v423: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:15.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:14 smithi161 ceph-mon[84406]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:15:15.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:14 smithi161 ceph-mon[84406]: pgmap v423: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:16.943 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:15:16 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:15:16] "GET /metrics HTTP/1.1" 200 38772 "" "Prometheus/2.33.4" 2024-06-01T02:15:16.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:16.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:16.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:16.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:16 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:15:16.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:16 smithi161 ceph-mon[84406]: pgmap v424: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-01T02:15:17.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:17.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:17.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:17.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:16 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:15:17.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:16 smithi116 ceph-mon[89502]: pgmap v424: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-01T02:15:17.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:17.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:17.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:17.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:16 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:15:17.044 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:16 smithi116 ceph-mon[100606]: pgmap v424: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-01T02:15:18.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:18.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:18.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:17 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:18.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:18.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:18.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:17 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:18.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:18.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:18.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:17 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:19.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:18 smithi116 ceph-mon[89502]: pgmap v425: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:19.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:15:19.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:15:19.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:15:19.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:15:19.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:15:19.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:15:19.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:15:19.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:15:19.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:19.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:19.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:15:19.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:15:19.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:18 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:19.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:18 smithi116 ceph-mon[100606]: pgmap v425: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:19.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:15:19.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:15:19.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:15:19.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:15:19.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:15:19.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:15:19.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:15:19.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:15:19.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:19.294 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:19.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:15:19.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:15:19.295 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:18 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:19.301 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:18 smithi161 ceph-mon[84406]: pgmap v425: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:19.301 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:15:19.301 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:15:19.301 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:15:19.301 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:15:19.301 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:15:19.301 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:15:19.301 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:15:19.301 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-01T02:15:19.301 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:19.301 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:19.301 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-01T02:15:19.302 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-01T02:15:19.302 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:18 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:20.247 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:20 smithi161 ceph-mon[84406]: Adjusting osd_memory_target on smithi116 to 2480M 2024-06-01T02:15:20.247 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:20 smithi161 ceph-mon[84406]: Adjusting osd_memory_target on smithi161 to 2736M 2024-06-01T02:15:20.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:20 smithi116 ceph-mon[89502]: Adjusting osd_memory_target on smithi116 to 2480M 2024-06-01T02:15:20.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:20 smithi116 ceph-mon[89502]: Adjusting osd_memory_target on smithi161 to 2736M 2024-06-01T02:15:20.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:20 smithi116 ceph-mon[100606]: Adjusting osd_memory_target on smithi116 to 2480M 2024-06-01T02:15:20.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:20 smithi116 ceph-mon[100606]: Adjusting osd_memory_target on smithi161 to 2736M 2024-06-01T02:15:21.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:21 smithi116 ceph-mon[89502]: pgmap v426: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-01T02:15:21.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:21 smithi116 ceph-mon[100606]: pgmap v426: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-01T02:15:21.420 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:21 smithi161 ceph-mon[84406]: pgmap v426: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-01T02:15:23.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:23 smithi116 ceph-mon[100606]: pgmap v427: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:23.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:23 smithi116 ceph-mon[89502]: pgmap v427: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:23.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:23 smithi161 ceph-mon[84406]: pgmap v427: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:25.218 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:25 smithi116 ceph-mon[89502]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:15:25.218 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:25 smithi116 ceph-mon[89502]: pgmap v428: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:25.218 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:25 smithi116 ceph-mon[100606]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:15:25.218 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:25 smithi116 ceph-mon[100606]: pgmap v428: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:25.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:25 smithi161 ceph-mon[84406]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:15:25.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:25 smithi161 ceph-mon[84406]: pgmap v428: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:27.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:27 smithi161 ceph-mon[84406]: pgmap v429: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-01T02:15:27.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:15:26 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:15:26] "GET /metrics HTTP/1.1" 200 38772 "" "Prometheus/2.33.4" 2024-06-01T02:15:27.234 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:27 smithi116 ceph-mon[100606]: pgmap v429: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-01T02:15:27.234 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:27 smithi116 ceph-mon[89502]: pgmap v429: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-01T02:15:29.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:29 smithi116 ceph-mon[89502]: pgmap v430: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:29.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:29 smithi116 ceph-mon[100606]: pgmap v430: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:29.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:29 smithi161 ceph-mon[84406]: pgmap v430: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:31.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:31 smithi116 ceph-mon[89502]: pgmap v431: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-01T02:15:31.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:31 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:15:31.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:31 smithi116 ceph-mon[100606]: pgmap v431: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-01T02:15:31.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:31 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:15:31.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:31 smithi161 ceph-mon[84406]: pgmap v431: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-01T02:15:31.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:31 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:15:33.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:33 smithi116 ceph-mon[89502]: pgmap v432: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:33.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:33 smithi116 ceph-mon[100606]: pgmap v432: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:33.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:33 smithi161 ceph-mon[84406]: pgmap v432: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:34.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:34 smithi161 ceph-mon[84406]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:15:34.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:34 smithi161 ceph-mon[84406]: pgmap v433: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:34.542 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:34 smithi116 ceph-mon[89502]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:15:34.542 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:34 smithi116 ceph-mon[89502]: pgmap v433: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:34.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:34 smithi116 ceph-mon[100606]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:15:34.543 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:34 smithi116 ceph-mon[100606]: pgmap v433: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:35.480 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph orch ps' 2024-06-01T02:15:36.783 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:36.784 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:36.784 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:36 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:36.784 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:36 smithi116 ceph-mon[100606]: pgmap v434: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-01T02:15:36.784 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:36.784 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:36.784 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:36 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:36.784 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:36 smithi116 ceph-mon[89502]: pgmap v434: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-01T02:15:37.119 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:37.119 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:37.119 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:36 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:37.119 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:36 smithi161 ceph-mon[84406]: pgmap v434: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-01T02:15:37.119 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:15:36 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:15:36] "GET /metrics HTTP/1.1" 200 38775 "" "Prometheus/2.33.4" 2024-06-01T02:15:37.547 INFO:teuthology.orchestra.run.smithi116.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-01T02:15:37.547 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi116 smithi116 running 1s ago 24m - - 2024-06-01T02:15:37.547 INFO:teuthology.orchestra.run.smithi116.stdout:agent.smithi161 smithi161 running 0s ago 24m - - 2024-06-01T02:15:37.547 INFO:teuthology.orchestra.run.smithi116.stdout:alertmanager.a smithi116 *:9093,9094 running (13m) 1s ago 18m - - 0.23.0 ba2b418f427c e0c297162ded 2024-06-01T02:15:37.547 INFO:teuthology.orchestra.run.smithi116.stdout:grafana.a smithi161 *:3000 running (13m) 0s ago 18m 55.5M - 8.3.5 dad864ee21e9 00b0a78ee88e 2024-06-01T02:15:37.547 INFO:teuthology.orchestra.run.smithi116.stdout:iscsi.foo.smithi116.gjuxzb smithi116 running (95s) 1s ago 17m - - 3.8 b045ac545f82 fe8075e0acba 2024-06-01T02:15:37.547 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.x smithi161 *:8443,9283,8765 running (12m) 0s ago 23m 543M - 19.0.0-2946-g3d89accf b045ac545f82 2d7fbb85e5f7 2024-06-01T02:15:37.547 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.y smithi116 *:8443,9283,8765 running (11m) 1s ago 25m - - 19.0.0-2946-g3d89accf b045ac545f82 bc9627c27a57 2024-06-01T02:15:37.547 INFO:teuthology.orchestra.run.smithi116.stdout:mon.a smithi116 running (10m) 1s ago 25m - 2048M 19.0.0-2946-g3d89accf b045ac545f82 b5e9b6f873ff 2024-06-01T02:15:37.547 INFO:teuthology.orchestra.run.smithi116.stdout:mon.b smithi161 running (9m) 0s ago 23m 59.9M 2048M 19.0.0-2946-g3d89accf b045ac545f82 aaa08c03cfc8 2024-06-01T02:15:37.548 INFO:teuthology.orchestra.run.smithi116.stdout:mon.c smithi116 running (10m) 1s ago 23m - 2048M 19.0.0-2946-g3d89accf b045ac545f82 bd0fb0e603fa 2024-06-01T02:15:37.548 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.a smithi116 *:9100 running (13m) 1s ago 19m - - 1.3.1 1dbe0e931976 e347748d17bb 2024-06-01T02:15:37.548 INFO:teuthology.orchestra.run.smithi116.stdout:node-exporter.b smithi161 *:9100 running (13m) 0s ago 19m 17.5M - 1.3.1 1dbe0e931976 6e66182f5a1a 2024-06-01T02:15:37.548 INFO:teuthology.orchestra.run.smithi116.stdout:osd.0 smithi116 running (8m) 1s ago 23m - 2480M 19.0.0-2946-g3d89accf b045ac545f82 b4313d121b9d 2024-06-01T02:15:37.548 INFO:teuthology.orchestra.run.smithi116.stdout:osd.1 smithi116 running (7m) 1s ago 22m - 2480M 19.0.0-2946-g3d89accf b045ac545f82 dccdbeac5aae 2024-06-01T02:15:37.548 INFO:teuthology.orchestra.run.smithi116.stdout:osd.2 smithi116 running (7m) 1s ago 22m - 2480M 19.0.0-2946-g3d89accf b045ac545f82 1ad95efda404 2024-06-01T02:15:37.548 INFO:teuthology.orchestra.run.smithi116.stdout:osd.3 smithi116 running (6m) 1s ago 21m - 2480M 19.0.0-2946-g3d89accf b045ac545f82 bd00ad2c2556 2024-06-01T02:15:37.548 INFO:teuthology.orchestra.run.smithi116.stdout:osd.4 smithi161 running (6m) 0s ago 21m 66.1M 2736M 19.0.0-2946-g3d89accf b045ac545f82 0084b9edf7a0 2024-06-01T02:15:37.548 INFO:teuthology.orchestra.run.smithi116.stdout:osd.5 smithi161 running (4m) 0s ago 21m 58.8M 2736M 19.0.0-2946-g3d89accf b045ac545f82 b3849cdeb707 2024-06-01T02:15:37.548 INFO:teuthology.orchestra.run.smithi116.stdout:osd.6 smithi161 running (4m) 0s ago 20m 53.6M 2736M 19.0.0-2946-g3d89accf b045ac545f82 6b87d9299c8d 2024-06-01T02:15:37.548 INFO:teuthology.orchestra.run.smithi116.stdout:osd.7 smithi161 running (3m) 0s ago 20m 60.1M 2736M 19.0.0-2946-g3d89accf b045ac545f82 400bb59c4249 2024-06-01T02:15:37.548 INFO:teuthology.orchestra.run.smithi116.stdout:prometheus.a smithi161 *:9095 running (12m) 0s ago 19m 79.1M - 2.33.4 514e6a882f6e e977f6ca1d73 2024-06-01T02:15:37.548 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi116.vzjrxj smithi116 *:8000 running (3m) 1s ago 17m - - 19.0.0-2946-g3d89accf b045ac545f82 376c2daccbd5 2024-06-01T02:15:37.549 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.foo.smithi161.ashqxz smithi161 *:8000 running (2m) 0s ago 17m 79.1M - 19.0.0-2946-g3d89accf b045ac545f82 6a0c3a3546be 2024-06-01T02:15:37.549 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi116.hokbmf smithi116 *:80 running (2m) 1s ago 17m - - 19.0.0-2946-g3d89accf b045ac545f82 3b0ab9880bcf 2024-06-01T02:15:37.549 INFO:teuthology.orchestra.run.smithi116.stdout:rgw.smpl.smithi161.axjlzw smithi161 *:80 running (2m) 0s ago 17m 76.8M - 19.0.0-2946-g3d89accf b045ac545f82 27533bf2d313 2024-06-01T02:15:38.207 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:38.208 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:38.208 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:38 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:38.208 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:38 smithi116 ceph-mon[89502]: from='client.45000 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:15:38.208 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:38 smithi116 ceph-mon[89502]: pgmap v435: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:38.208 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:38.208 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:38.208 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:38 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:38.209 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:38 smithi116 ceph-mon[100606]: from='client.45000 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:15:38.209 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:38 smithi116 ceph-mon[100606]: pgmap v435: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:38.230 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph versions' 2024-06-01T02:15:38.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:38.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:38.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:38 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:38.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:38 smithi161 ceph-mon[84406]: from='client.45000 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:15:38.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:38 smithi161 ceph-mon[84406]: pgmap v435: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:40.316 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:15:40.316 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": { 2024-06-01T02:15:40.316 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 3 2024-06-01T02:15:40.316 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:15:40.316 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": { 2024-06-01T02:15:40.316 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-01T02:15:40.316 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:15:40.316 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": { 2024-06-01T02:15:40.316 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 8 2024-06-01T02:15:40.317 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:15:40.317 INFO:teuthology.orchestra.run.smithi116.stdout: "rgw": { 2024-06-01T02:15:40.317 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 4 2024-06-01T02:15:40.317 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-06-01T02:15:40.317 INFO:teuthology.orchestra.run.smithi116.stdout: "overall": { 2024-06-01T02:15:40.317 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 17 2024-06-01T02:15:40.317 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-06-01T02:15:40.317 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:15:41.007 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph orch upgrade status' 2024-06-01T02:15:41.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:40 smithi116 ceph-mon[89502]: pgmap v436: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-01T02:15:41.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:40 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/1345168210' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:15:41.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:40 smithi116 ceph-mon[100606]: pgmap v436: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-01T02:15:41.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:40 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/1345168210' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:15:41.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:40 smithi161 ceph-mon[84406]: pgmap v436: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-01T02:15:41.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:40 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/1345168210' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:15:43.111 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:15:43.112 INFO:teuthology.orchestra.run.smithi116.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-01T02:15:43.112 INFO:teuthology.orchestra.run.smithi116.stdout: "in_progress": true, 2024-06-01T02:15:43.112 INFO:teuthology.orchestra.run.smithi116.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-01T02:15:43.112 INFO:teuthology.orchestra.run.smithi116.stdout: "services_complete": [ 2024-06-01T02:15:43.112 INFO:teuthology.orchestra.run.smithi116.stdout: "iscsi", 2024-06-01T02:15:43.112 INFO:teuthology.orchestra.run.smithi116.stdout: "mon", 2024-06-01T02:15:43.112 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr", 2024-06-01T02:15:43.112 INFO:teuthology.orchestra.run.smithi116.stdout: "rgw", 2024-06-01T02:15:43.112 INFO:teuthology.orchestra.run.smithi116.stdout: "osd" 2024-06-01T02:15:43.113 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-06-01T02:15:43.113 INFO:teuthology.orchestra.run.smithi116.stdout: "progress": "18/23 daemons upgraded", 2024-06-01T02:15:43.113 INFO:teuthology.orchestra.run.smithi116.stdout: "message": "Error: UPGRADE_REDEPLOY_DAEMON: Upgrading daemon iscsi.foo.smithi116.gjuxzb on host smithi116 failed.", 2024-06-01T02:15:43.113 INFO:teuthology.orchestra.run.smithi116.stdout: "is_paused": true 2024-06-01T02:15:43.113 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:15:43.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:42 smithi116 ceph-mon[89502]: pgmap v437: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:43.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:42 smithi116 ceph-mon[100606]: pgmap v437: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:43.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:42 smithi161 ceph-mon[84406]: pgmap v437: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:43.833 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph health detail' 2024-06-01T02:15:45.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:44 smithi116 ceph-mon[89502]: from='client.64993 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:15:45.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:44 smithi116 ceph-mon[89502]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:15:45.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:44 smithi116 ceph-mon[89502]: pgmap v438: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:45.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:44 smithi116 ceph-mon[100606]: from='client.64993 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:15:45.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:44 smithi116 ceph-mon[100606]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:15:45.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:44 smithi116 ceph-mon[100606]: pgmap v438: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:45.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:44 smithi161 ceph-mon[84406]: from='client.64993 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:15:45.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:44 smithi161 ceph-mon[84406]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:15:45.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:44 smithi161 ceph-mon[84406]: pgmap v438: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:46.219 INFO:teuthology.orchestra.run.smithi116.stdout:HEALTH_WARN Upgrading daemon iscsi.foo.smithi116.gjuxzb on host smithi116 failed. 2024-06-01T02:15:46.219 INFO:teuthology.orchestra.run.smithi116.stdout:[WRN] UPGRADE_REDEPLOY_DAEMON: Upgrading daemon iscsi.foo.smithi116.gjuxzb on host smithi116 failed. 2024-06-01T02:15:46.219 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrade daemon: iscsi.foo.smithi116.gjuxzb: cephadm exited with an error code: 1, stderr: Redeploy daemon iscsi.foo.smithi116.gjuxzb ... 2024-06-01T02:15:46.219 INFO:teuthology.orchestra.run.smithi116.stdout:Creating ceph-iscsi config... 2024-06-01T02:15:46.219 INFO:teuthology.orchestra.run.smithi116.stdout:Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/iscsi-gateway.cfg 2024-06-01T02:15:46.219 INFO:teuthology.orchestra.run.smithi116.stdout:Write file: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/iscsi.foo.smithi116.gjuxzb/tcmu-runner-entrypoint.sh 2024-06-01T02:15:46.219 INFO:teuthology.orchestra.run.smithi116.stdout:Failed to trim old cgroups /sys/fs/cgroup/system.slice/system-ceph\x2d393f964a\x2d1fb9\x2d11ef\x2dbc9b\x2dc7b262605968.slice/ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service 2024-06-01T02:15:46.219 INFO:teuthology.orchestra.run.smithi116.stdout:Non-zero exit code 1 from systemctl start ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb 2024-06-01T02:15:46.219 INFO:teuthology.orchestra.run.smithi116.stdout:systemctl: stderr Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:15:46.220 INFO:teuthology.orchestra.run.smithi116.stdout:systemctl: stderr See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:15:46.220 INFO:teuthology.orchestra.run.smithi116.stdout:Traceback (most recent call last): 2024-06-01T02:15:46.220 INFO:teuthology.orchestra.run.smithi116.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-06-01T02:15:46.220 INFO:teuthology.orchestra.run.smithi116.stdout: return _run_code(code, main_globals, None, 2024-06-01T02:15:46.220 INFO:teuthology.orchestra.run.smithi116.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-06-01T02:15:46.220 INFO:teuthology.orchestra.run.smithi116.stdout: exec(code, run_globals) 2024-06-01T02:15:46.220 INFO:teuthology.orchestra.run.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5618, in 2024-06-01T02:15:46.220 INFO:teuthology.orchestra.run.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 5606, in main 2024-06-01T02:15:46.220 INFO:teuthology.orchestra.run.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3096, in command_deploy_from 2024-06-01T02:15:46.220 INFO:teuthology.orchestra.run.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3131, in _common_deploy 2024-06-01T02:15:46.220 INFO:teuthology.orchestra.run.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 3151, in _deploy_daemon_container 2024-06-01T02:15:46.220 INFO:teuthology.orchestra.run.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 1087, in deploy_daemon 2024-06-01T02:15:46.220 INFO:teuthology.orchestra.run.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/__main__.py", line 1223, in deploy_daemon_units 2024-06-01T02:15:46.220 INFO:teuthology.orchestra.run.smithi116.stdout: File "/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/cephadm.8cc035ff3971b2e34a337b7a71505dc9e2ace82f3807fb6c384c059d64bde4dc/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-06-01T02:15:46.221 INFO:teuthology.orchestra.run.smithi116.stdout:RuntimeError: Failed command: systemctl start ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb: Job for ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service failed because the control process exited with error code. 2024-06-01T02:15:46.221 INFO:teuthology.orchestra.run.smithi116.stdout:See "systemctl status ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" and "journalctl -xeu ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@iscsi.foo.smithi116.gjuxzb.service" for details. 2024-06-01T02:15:46.221 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:45 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:15:46.221 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:45 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:15:46.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:45 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:15:47.040 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-06-01T02:15:47.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:46 smithi161 ceph-mon[84406]: pgmap v439: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:47.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:46 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/3632388110' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:15:47.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:15:46 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:15:46] "GET /metrics HTTP/1.1" 200 38775 "" "Prometheus/2.33.4" 2024-06-01T02:15:47.251 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:46 smithi116 ceph-mon[89502]: pgmap v439: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:47.251 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:46 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/3632388110' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:15:47.251 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:46 smithi116 ceph-mon[100606]: pgmap v439: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:47.251 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:46 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/3632388110' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-01T02:15:49.130 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:48 smithi116 ceph-mon[89502]: pgmap v440: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:49.131 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:48 smithi116 ceph-mon[100606]: pgmap v440: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:49.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:48 smithi161 ceph-mon[84406]: pgmap v440: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:50.672 INFO:teuthology.orchestra.run.smithi116.stdout:true 2024-06-01T02:15:51.249 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:50 smithi116 ceph-mon[100606]: pgmap v441: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:51.249 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:50 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/4172245221' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:15:51.249 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:50 smithi116 ceph-mon[89502]: pgmap v441: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:51.249 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:50 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/4172245221' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:15:51.274 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-06-01T02:15:51.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:50 smithi161 ceph-mon[84406]: pgmap v441: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:51.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:50 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/4172245221' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:15:53.198 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:52 smithi116 ceph-mon[89502]: pgmap v442: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:53.198 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:52 smithi116 ceph-mon[100606]: pgmap v442: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:53.246 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:52 smithi161 ceph-mon[84406]: pgmap v442: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:53.426 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)" 2024-06-01T02:15:54.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:53 smithi116 ceph-mon[89502]: from='client.? 172.21.15.116:0/172590417' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:15:54.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:53 smithi116 ceph-mon[100606]: from='client.? 172.21.15.116:0/172590417' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:15:54.071 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph orch ls | grep '"'"'^osd '"'"'' 2024-06-01T02:15:54.261 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:53 smithi161 ceph-mon[84406]: from='client.? 172.21.15.116:0/172590417' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-01T02:15:55.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:54 smithi116 ceph-mon[89502]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:15:55.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:54 smithi116 ceph-mon[89502]: pgmap v443: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:55.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:54 smithi116 ceph-mon[100606]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:15:55.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:54 smithi116 ceph-mon[100606]: pgmap v443: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:55.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:54 smithi161 ceph-mon[84406]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:15:55.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:54 smithi161 ceph-mon[84406]: pgmap v443: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:56.182 INFO:teuthology.orchestra.run.smithi116.stdout:osd 8 19s ago - 2024-06-01T02:15:56.776 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-01T02:15:56.787 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi116.front.sepia.ceph.com 2024-06-01T02:15:56.787 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch upgrade ls' 2024-06-01T02:15:57.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:57.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:57.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:56 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:57.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:56 smithi116 ceph-mon[89502]: pgmap v444: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:57.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:57.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:57.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:56 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:57.043 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:56 smithi116 ceph-mon[100606]: pgmap v444: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:57.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:15:56 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:15:56] "GET /metrics HTTP/1.1" 200 38775 "" "Prometheus/2.33.4" 2024-06-01T02:15:57.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:57.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:57.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:56 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:57.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:56 smithi161 ceph-mon[84406]: pgmap v444: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:15:58.026 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:57 smithi116 ceph-mon[100606]: from='client.45036 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:15:58.026 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:57 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:58.027 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:57 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:58.027 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:57 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:58.027 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:57 smithi116 ceph-mon[89502]: from='client.45036 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:15:58.027 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:58.028 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:58.028 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:57 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:58.193 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:57 smithi161 ceph-mon[84406]: from='client.45036 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:15:58.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:57 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:58.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:57 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:58.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:57 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' 2024-06-01T02:15:59.138 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:58 smithi161 ceph-mon[84406]: pgmap v445: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:59.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:58 smithi116 ceph-mon[89502]: pgmap v445: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:59.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:58 smithi116 ceph-mon[100606]: pgmap v445: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:15:59.484 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-06-01T02:15:59.484 INFO:teuthology.orchestra.run.smithi116.stdout: "image": "quay.io/ceph/ceph", 2024-06-01T02:15:59.484 INFO:teuthology.orchestra.run.smithi116.stdout: "registry": "quay.io", 2024-06-01T02:15:59.484 INFO:teuthology.orchestra.run.smithi116.stdout: "bare_image": "ceph/ceph", 2024-06-01T02:15:59.484 INFO:teuthology.orchestra.run.smithi116.stdout: "versions": [] 2024-06-01T02:15:59.484 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-06-01T02:15:59.943 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:15:59 smithi161 ceph-mon[84406]: from='client.65023 -' entity='client.admin' cmd=[{"prefix": "orch upgrade ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:16:00.042 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:15:59 smithi116 ceph-mon[100606]: from='client.65023 -' entity='client.admin' cmd=[{"prefix": "orch upgrade ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:16:00.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:15:59 smithi116 ceph-mon[89502]: from='client.65023 -' entity='client.admin' cmd=[{"prefix": "orch upgrade ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:16:00.276 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch upgrade ls --image quay.io/ceph/ceph --show-all-versions | grep 16.2.0' 2024-06-01T02:16:01.131 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:16:00 smithi161 ceph-mon[84406]: pgmap v446: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:16:01.131 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:16:00 smithi161 ceph-mon[84406]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:16:01.224 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:16:00 smithi116 ceph-mon[100606]: pgmap v446: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:16:01.224 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:16:00 smithi116 ceph-mon[100606]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:16:01.224 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:16:00 smithi116 ceph-mon[89502]: pgmap v446: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:16:01.224 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:16:00 smithi116 ceph-mon[89502]: from='mgr.25280 172.21.15.161:0/3064007507' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-01T02:16:03.068 INFO:teuthology.orchestra.run.smithi116.stdout: "16.2.0", 2024-06-01T02:16:03.292 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:16:02 smithi116 ceph-mon[89502]: pgmap v447: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:16:03.293 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:16:02 smithi116 ceph-mon[100606]: pgmap v447: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:16:03.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:16:02 smithi161 ceph-mon[84406]: pgmap v447: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:16:03.812 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch upgrade ls --image quay.io/ceph/ceph --tags | grep v16.2.2' 2024-06-01T02:16:04.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:16:03 smithi116 ceph-mon[89502]: from='client.65029 -' entity='client.admin' cmd=[{"prefix": "orch upgrade ls", "image": "quay.io/ceph/ceph", "show_all_versions": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:16:04.042 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:16:03 smithi116 ceph-mon[100606]: from='client.65029 -' entity='client.admin' cmd=[{"prefix": "orch upgrade ls", "image": "quay.io/ceph/ceph", "show_all_versions": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:16:04.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:16:03 smithi161 ceph-mon[84406]: from='client.65029 -' entity='client.admin' cmd=[{"prefix": "orch upgrade ls", "image": "quay.io/ceph/ceph", "show_all_versions": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:16:05.222 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:16:04 smithi116 ceph-mon[89502]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:16:05.222 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:16:04 smithi116 ceph-mon[89502]: pgmap v448: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:16:05.222 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:16:04 smithi116 ceph-mon[100606]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:16:05.223 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:16:04 smithi116 ceph-mon[100606]: pgmap v448: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:16:05.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:16:04 smithi161 ceph-mon[84406]: from='client.44964 -' entity='client.iscsi.foo.smithi116.gjuxzb' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-01T02:16:05.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:16:04 smithi161 ceph-mon[84406]: pgmap v448: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:16:06.748 INFO:teuthology.orchestra.run.smithi116.stdout: "v16.2.2", 2024-06-01T02:16:06.748 INFO:teuthology.orchestra.run.smithi116.stdout: "v16.2.2-20210505", 2024-06-01T02:16:07.037 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:16:06 smithi116 ceph-mon[89502]: pgmap v449: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:16:07.037 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:16:06 smithi116 ceph-mon[89502]: from='client.45054 -' entity='client.admin' cmd=[{"prefix": "orch upgrade ls", "image": "quay.io/ceph/ceph", "tags": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:16:07.038 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:16:06 smithi116 ceph-mon[100606]: pgmap v449: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:16:07.038 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:16:06 smithi116 ceph-mon[100606]: from='client.45054 -' entity='client.admin' cmd=[{"prefix": "orch upgrade ls", "image": "quay.io/ceph/ceph", "tags": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:16:07.193 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:16:06 smithi161 ceph-mon[84406]: pgmap v449: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-01T02:16:07.194 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:16:06 smithi161 ceph-mon[84406]: from='client.45054 -' entity='client.admin' cmd=[{"prefix": "orch upgrade ls", "image": "quay.io/ceph/ceph", "tags": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-01T02:16:07.194 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:16:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: ::ffff:172.21.15.161 - - [01/Jun/2024:02:16:06] "GET /metrics HTTP/1.1" 200 38776 "" "Prometheus/2.33.4" 2024-06-01T02:16:08.523 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-06-01T02:16:08.532 INFO:tasks.cephadm:Teardown begin 2024-06-01T02:16:08.532 DEBUG:teuthology.orchestra.run.smithi116:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-01T02:16:08.562 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-01T02:16:08.598 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-06-01T02:16:08.599 DEBUG:teuthology.orchestra.run.smithi116:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-01T02:16:08.617 DEBUG:teuthology.orchestra.run.smithi161:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-01T02:16:08.656 INFO:tasks.cephadm:Stopping all daemons... 2024-06-01T02:16:08.657 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-06-01T02:16:08.657 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mon.a 2024-06-01T02:16:08.750 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:16:08 smithi116 systemd[1]: Stopping Ceph mon.a for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:16:09.042 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:16:08 smithi116 ceph-mon[100606]: pgmap v450: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:16:09.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:16:08 smithi116 ceph-mon[89502]: pgmap v450: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:16:09.250 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:16:08 smithi161 ceph-mon[84406]: pgmap v450: 161 pgs: 161 active+clean; 585 KiB data, 296 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-01T02:16:09.251 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:16:09 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: [01/Jun/2024:02:16:09] ENGINE Bus STOPPING 2024-06-01T02:16:09.251 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:16:09 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: [01/Jun/2024:02:16:09] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-06-01T02:16:09.251 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:16:09 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: [01/Jun/2024:02:16:09] ENGINE Bus STOPPED 2024-06-01T02:16:09.251 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:16:09 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: [01/Jun/2024:02:16:09] ENGINE Bus STARTING 2024-06-01T02:16:09.251 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:16:09 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: CherryPy Checker: 2024-06-01T02:16:09.251 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:16:09 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: The Application mounted at '' has an empty config. 2024-06-01T02:16:09.251 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:16:09 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:16:09.522 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:16:09 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a[89498]: 2024-06-01T02:16:09.133+0000 7eff1d262700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-01T02:16:09.522 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:16:09 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a[89498]: 2024-06-01T02:16:09.133+0000 7eff1d262700 -1 mon.a@0(leader) e4 *** Got Signal Terminated *** 2024-06-01T02:16:09.523 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:16:09 smithi116 podman[162540]: 2024-06-01 02:16:09.378849819 +0000 UTC m=+0.384667995 container died b5e9b6f873ff4557c245c9d8120ac3cf97f27788951a06beb9a5d8a39ee6374b (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-06-01T02:16:09.693 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:16:09 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: [01/Jun/2024:02:16:09] ENGINE Serving on http://:::9283 2024-06-01T02:16:09.694 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:16:09 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: [01/Jun/2024:02:16:09] ENGINE Bus STARTED 2024-06-01T02:16:10.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:16:09 smithi116 podman[162540]: 2024-06-01 02:16:09.783423347 +0000 UTC m=+0.789241531 container cleanup b5e9b6f873ff4557c245c9d8120ac3cf97f27788951a06beb9a5d8a39ee6374b (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-06-01T02:16:10.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:16:09 smithi116 bash[162540]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a 2024-06-01T02:16:10.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Jun 01 02:16:10 smithi116 podman[162579]: 2024-06-01 02:16:10.001332447 +0000 UTC m=+0.625965229 container remove b5e9b6f873ff4557c245c9d8120ac3cf97f27788951a06beb9a5d8a39ee6374b (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-01T02:16:11.042 DEBUG:teuthology.orchestra.run.smithi116:> sudo pkill -f 'journalctl -f -n 0 -u ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mon.a.service' 2024-06-01T02:16:11.089 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-01T02:16:11.089 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-06-01T02:16:11.089 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-06-01T02:16:11.090 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mon.c 2024-06-01T02:16:11.292 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:16:11 smithi116 systemd[1]: Stopping Ceph mon.c for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:16:11.737 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:16:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-c[100602]: 2024-06-01T02:16:11.399+0000 7f27f82d3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-01T02:16:11.737 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:16:11 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-c[100602]: 2024-06-01T02:16:11.399+0000 7f27f82d3700 -1 mon.c@1(peon) e4 *** Got Signal Terminated *** 2024-06-01T02:16:11.737 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:16:11 smithi116 podman[162712]: 2024-06-01 02:16:11.492322597 +0000 UTC m=+0.212127638 container died bd0fb0e603fa7534f39c8dab091512a8cb6bad8a3d9a396935fd05acecd63c06 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-c, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-06-01T02:16:12.012 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:16:11 smithi116 podman[162712]: 2024-06-01 02:16:11.84182543 +0000 UTC m=+0.561630436 container cleanup bd0fb0e603fa7534f39c8dab091512a8cb6bad8a3d9a396935fd05acecd63c06 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS) 2024-06-01T02:16:12.013 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:16:11 smithi116 bash[162712]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-c 2024-06-01T02:16:12.263 INFO:journalctl@ceph.mon.c.smithi116.stdout:Jun 01 02:16:12 smithi116 podman[162724]: 2024-06-01 02:16:12.018163228 +0000 UTC m=+0.521425871 container remove bd0fb0e603fa7534f39c8dab091512a8cb6bad8a3d9a396935fd05acecd63c06 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-01T02:16:12.266 DEBUG:teuthology.orchestra.run.smithi116:> sudo pkill -f 'journalctl -f -n 0 -u ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mon.c.service' 2024-06-01T02:16:12.301 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-01T02:16:12.302 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-06-01T02:16:12.302 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-06-01T02:16:12.302 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mon.b 2024-06-01T02:16:12.603 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:16:12 smithi161 systemd[1]: Stopping Ceph mon.b for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:16:12.867 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:16:12 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-b[84402]: 2024-06-01T02:16:12.607+0000 7f931b171700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-01T02:16:12.868 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:16:12 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-b[84402]: 2024-06-01T02:16:12.607+0000 7f931b171700 -1 mon.b@2(peon) e4 *** Got Signal Terminated *** 2024-06-01T02:16:13.170 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:16:12 smithi161 podman[147915]: 2024-06-01 02:16:12.873031304 +0000 UTC m=+0.369726002 container died aaa08c03cfc87c81be34af9daf0b4aeefcb8473cbe3ebb93fb68eebda6ff9ca4 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-06-01T02:16:13.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:16:13 smithi161 podman[147915]: 2024-06-01 02:16:13.358232547 +0000 UTC m=+0.854927266 container cleanup aaa08c03cfc87c81be34af9daf0b4aeefcb8473cbe3ebb93fb68eebda6ff9ca4 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-b, RELEASE=squid-3d89acc, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0) 2024-06-01T02:16:13.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jun 01 02:16:13 smithi161 bash[147915]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mon-b 2024-06-01T02:16:13.774 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mon.b.service' 2024-06-01T02:16:13.835 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-01T02:16:13.836 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-06-01T02:16:13.836 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-06-01T02:16:13.836 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mgr.y 2024-06-01T02:16:14.042 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:16:13 smithi116 systemd[1]: Stopping Ceph mgr.y for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:16:14.603 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:16:14 smithi116 podman[163072]: 2024-06-01 02:16:14.356243469 +0000 UTC m=+0.132467553 container died bc9627c27a5790b781a019c6e738e7cd47dfca82f60f338b37f8fa5f1058fcfd (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-06-01T02:16:14.912 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:16:14 smithi116 podman[163072]: 2024-06-01 02:16:14.691171531 +0000 UTC m=+0.467395614 container cleanup bc9627c27a5790b781a019c6e738e7cd47dfca82f60f338b37f8fa5f1058fcfd (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-01T02:16:14.912 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:16:14 smithi116 bash[163072]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y 2024-06-01T02:16:15.166 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:16:14 smithi116 podman[163090]: 2024-06-01 02:16:14.917711926 +0000 UTC m=+0.556484129 container remove bc9627c27a5790b781a019c6e738e7cd47dfca82f60f338b37f8fa5f1058fcfd (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-06-01T02:16:15.166 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Jun 01 02:16:14 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-06-01T02:16:15.181 DEBUG:teuthology.orchestra.run.smithi116:> sudo pkill -f 'journalctl -f -n 0 -u ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mgr.y.service' 2024-06-01T02:16:15.217 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-01T02:16:15.217 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-06-01T02:16:15.217 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-06-01T02:16:15.217 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mgr.x 2024-06-01T02:16:15.550 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:16:15 smithi161 systemd[1]: Stopping Ceph mgr.x for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:16:15.843 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:16:15 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:16:15.553+0000 7fcf1206a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-01T02:16:15.844 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:16:15 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x[59637]: 2024-06-01T02:16:15.553+0000 7fcf1206a700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-06-01T02:16:15.844 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:16:15 smithi161 podman[148106]: 2024-06-01 02:16:15.596403449 +0000 UTC m=+0.164615431 container died 2d7fbb85e5f748fda5cfc7eb53872a5dba1bfcd91df68f9ef166a946e3f4640a (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0) 2024-06-01T02:16:16.193 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:16:16 smithi161 podman[148106]: 2024-06-01 02:16:16.022247172 +0000 UTC m=+0.590459145 container cleanup 2d7fbb85e5f748fda5cfc7eb53872a5dba1bfcd91df68f9ef166a946e3f4640a (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-01T02:16:16.193 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:16:16 smithi161 bash[148106]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x 2024-06-01T02:16:16.501 INFO:journalctl@ceph.mgr.x.smithi161.stdout:Jun 01 02:16:16 smithi161 podman[148151]: 2024-06-01 02:16:16.240796226 +0000 UTC m=+0.643399754 container remove 2d7fbb85e5f748fda5cfc7eb53872a5dba1bfcd91df68f9ef166a946e3f4640a (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-06-01T02:16:16.520 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@mgr.x.service' 2024-06-01T02:16:16.558 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-01T02:16:16.559 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-06-01T02:16:16.559 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-06-01T02:16:16.559 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.0 2024-06-01T02:16:16.792 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:16:16 smithi116 systemd[1]: Stopping Ceph osd.0 for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:16:18.042 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:16:17 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0[114041]: 2024-06-01T02:16:17.673+0000 7f9c16ef3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-01T02:16:18.043 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:16:17 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0[114041]: 2024-06-01T02:16:17.673+0000 7f9c16ef3700 -1 osd.0 196 *** Got signal Terminated *** 2024-06-01T02:16:18.043 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:16:17 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0[114041]: 2024-06-01T02:16:17.673+0000 7f9c16ef3700 -1 osd.0 196 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-01T02:16:23.521 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:16:23 smithi116 podman[163528]: 2024-06-01 02:16:23.205072976 +0000 UTC m=+6.262630919 container died b4313d121b9d3a7150b6144d61c0216ab9001ea5798e2726752809ed28f7977d (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5) 2024-06-01T02:16:23.792 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:16:23 smithi116 podman[163528]: 2024-06-01 02:16:23.716087403 +0000 UTC m=+6.773645354 container cleanup b4313d121b9d3a7150b6144d61c0216ab9001ea5798e2726752809ed28f7977d (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc) 2024-06-01T02:16:23.792 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:16:23 smithi116 bash[163528]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0 2024-06-01T02:16:24.194 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:16:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:16:23.709Z caller=refresh.go:98 level=error component="discovery manager notify" discovery=http msg="Unable to refresh target groups" err="Get \"http://172.21.15.161:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.161:8765: connect: connection refused" 2024-06-01T02:16:24.194 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:16:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:16:23.710Z caller=refresh.go:98 level=error component="discovery manager scrape" discovery=http msg="Unable to refresh target groups" err="Get \"http://172.21.15.161:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.161:8765: connect: connection refused" 2024-06-01T02:16:24.194 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:16:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:16:23.713Z caller=refresh.go:98 level=error component="discovery manager scrape" discovery=http msg="Unable to refresh target groups" err="Get \"http://172.21.15.161:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.161:8765: connect: connection refused" 2024-06-01T02:16:24.194 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:16:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:16:23.713Z caller=refresh.go:98 level=error component="discovery manager scrape" discovery=http msg="Unable to refresh target groups" err="Get \"http://172.21.15.161:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.161:8765: connect: connection refused" 2024-06-01T02:16:24.194 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:16:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:16:23.713Z caller=refresh.go:98 level=error component="discovery manager scrape" discovery=http msg="Unable to refresh target groups" err="Get \"http://172.21.15.161:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.161:8765: connect: connection refused" 2024-06-01T02:16:24.288 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:16:23 smithi116 podman[163594]: 2024-06-01 02:16:23.967418819 +0000 UTC m=+0.756373890 container remove b4313d121b9d3a7150b6144d61c0216ab9001ea5798e2726752809ed28f7977d (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-06-01T02:16:24.542 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:16:24 smithi116 podman[163687]: 2024-06-01 02:16:24.293285233 +0000 UTC m=+0.113118550 container create 3b8a93575b4dc56912379ae0761efd1c0cd75dbf4bc4b214287eb5e12dcba510 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-deactivate, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-01T02:16:24.542 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:16:24 smithi116 podman[163687]: 2024-06-01 02:16:24.197499056 +0000 UTC m=+0.017332400 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:16:24.543 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:16:24 smithi116 podman[163687]: 2024-06-01 02:16:24.521438868 +0000 UTC m=+0.341272199 container init 3b8a93575b4dc56912379ae0761efd1c0cd75dbf4bc4b214287eb5e12dcba510 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-06-01T02:16:24.543 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:16:24 smithi116 podman[163687]: 2024-06-01 02:16:24.525168049 +0000 UTC m=+0.345001372 container start 3b8a93575b4dc56912379ae0761efd1c0cd75dbf4bc4b214287eb5e12dcba510 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=) 2024-06-01T02:16:25.042 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:16:24 smithi116 podman[163687]: 2024-06-01 02:16:24.621263272 +0000 UTC m=+0.441096589 container attach 3b8a93575b4dc56912379ae0761efd1c0cd75dbf4bc4b214287eb5e12dcba510 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-06-01T02:16:25.042 INFO:journalctl@ceph.osd.0.smithi116.stdout:Jun 01 02:16:24 smithi116 podman[163687]: 2024-06-01 02:16:24.814702867 +0000 UTC m=+0.634536184 container died 3b8a93575b4dc56912379ae0761efd1c0cd75dbf4bc4b214287eb5e12dcba510 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-06-01T02:16:25.520 DEBUG:teuthology.orchestra.run.smithi116:> sudo pkill -f 'journalctl -f -n 0 -u ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.0.service' 2024-06-01T02:16:25.563 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-01T02:16:25.564 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-06-01T02:16:25.564 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-06-01T02:16:25.564 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.1 2024-06-01T02:16:25.792 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:16:25 smithi116 systemd[1]: Stopping Ceph osd.1 for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:16:26.292 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:16:25 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1[119010]: 2024-06-01T02:16:25.904+0000 7f8243d92700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-01T02:16:26.292 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:16:25 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1[119010]: 2024-06-01T02:16:25.904+0000 7f8243d92700 -1 osd.1 196 *** Got signal Terminated *** 2024-06-01T02:16:26.292 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:16:25 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1[119010]: 2024-06-01T02:16:25.904+0000 7f8243d92700 -1 osd.1 196 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-01T02:16:31.605 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:16:31 smithi116 podman[163792]: 2024-06-01 02:16:31.314301215 +0000 UTC m=+5.512809973 container died dccdbeac5aaed8e1335f250712c904f1e1cb75ba98c37420ef233be25adcfd36 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-3d89acc, ceph=True) 2024-06-01T02:16:31.955 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:16:31 smithi116 podman[163792]: 2024-06-01 02:16:31.709137096 +0000 UTC m=+5.907645853 container cleanup dccdbeac5aaed8e1335f250712c904f1e1cb75ba98c37420ef233be25adcfd36 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-06-01T02:16:31.955 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:16:31 smithi116 bash[163792]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1 2024-06-01T02:16:32.292 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:16:31 smithi116 podman[163930]: 2024-06-01 02:16:31.960454809 +0000 UTC m=+0.641997094 container remove dccdbeac5aaed8e1335f250712c904f1e1cb75ba98c37420ef233be25adcfd36 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-01T02:16:32.591 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:16:32 smithi116 podman[164022]: 2024-06-01 02:16:32.203785238 +0000 UTC m=+0.017288703 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:16:32.591 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:16:32 smithi116 podman[164022]: 2024-06-01 02:16:32.302936833 +0000 UTC m=+0.116440273 container create cafb93e5c5281447d185a14692c8172d8486d38290adba8b15cc6e8ea152e985 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True) 2024-06-01T02:16:32.592 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:16:32 smithi116 podman[164022]: 2024-06-01 02:16:32.530560759 +0000 UTC m=+0.344064198 container init cafb93e5c5281447d185a14692c8172d8486d38290adba8b15cc6e8ea152e985 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-06-01T02:16:32.592 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:16:32 smithi116 podman[164022]: 2024-06-01 02:16:32.534113483 +0000 UTC m=+0.347616939 container start cafb93e5c5281447d185a14692c8172d8486d38290adba8b15cc6e8ea152e985 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-06-01T02:16:32.592 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:16:32 smithi116 podman[164022]: 2024-06-01 02:16:32.597212541 +0000 UTC m=+0.410716019 container attach cafb93e5c5281447d185a14692c8172d8486d38290adba8b15cc6e8ea152e985 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-06-01T02:16:33.042 INFO:journalctl@ceph.osd.1.smithi116.stdout:Jun 01 02:16:32 smithi116 podman[164022]: 2024-06-01 02:16:32.824342757 +0000 UTC m=+0.637846212 container died cafb93e5c5281447d185a14692c8172d8486d38290adba8b15cc6e8ea152e985 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240528) 2024-06-01T02:16:33.470 DEBUG:teuthology.orchestra.run.smithi116:> sudo pkill -f 'journalctl -f -n 0 -u ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.1.service' 2024-06-01T02:16:33.512 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-01T02:16:33.512 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-06-01T02:16:33.512 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-06-01T02:16:33.512 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.2 2024-06-01T02:16:33.792 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:16:33 smithi116 systemd[1]: Stopping Ceph osd.2 for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:16:34.292 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:16:33 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2[123906]: 2024-06-01T02:16:33.879+0000 7fae3de13700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-01T02:16:34.292 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:16:33 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2[123906]: 2024-06-01T02:16:33.879+0000 7fae3de13700 -1 osd.2 196 *** Got signal Terminated *** 2024-06-01T02:16:34.292 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:16:33 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2[123906]: 2024-06-01T02:16:33.879+0000 7fae3de13700 -1 osd.2 196 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-01T02:16:39.542 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:16:39 smithi116 podman[164129]: 2024-06-01 02:16:39.349300024 +0000 UTC m=+5.607187131 container died 1ad95efda4043f841727af00c02ba41530144fe0b34aed28f3efbc14296ed566 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS) 2024-06-01T02:16:40.029 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:16:39 smithi116 podman[164129]: 2024-06-01 02:16:39.766752501 +0000 UTC m=+6.024639594 container cleanup 1ad95efda4043f841727af00c02ba41530144fe0b34aed28f3efbc14296ed566 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-06-01T02:16:40.029 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:16:39 smithi116 bash[164129]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2 2024-06-01T02:16:40.292 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:16:40 smithi116 podman[164637]: 2024-06-01 02:16:40.034772401 +0000 UTC m=+0.681654004 container remove 1ad95efda4043f841727af00c02ba41530144fe0b34aed28f3efbc14296ed566 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-06-01T02:16:42.161 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:16:42 smithi116 podman[164906]: 2024-06-01 02:16:41.970170585 +0000 UTC m=+0.017294294 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:16:42.161 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:16:42 smithi116 podman[164906]: 2024-06-01 02:16:42.068679715 +0000 UTC m=+0.115803460 container create 743c8cad2b0c156bd75a2bc3d9f88607192e2c133f567c571ad075551f751444 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2-deactivate, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-06-01T02:16:42.542 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:16:42 smithi116 podman[164906]: 2024-06-01 02:16:42.271293047 +0000 UTC m=+0.318416754 container init 743c8cad2b0c156bd75a2bc3d9f88607192e2c133f567c571ad075551f751444 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-01T02:16:42.543 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:16:42 smithi116 podman[164906]: 2024-06-01 02:16:42.274658577 +0000 UTC m=+0.321782284 container start 743c8cad2b0c156bd75a2bc3d9f88607192e2c133f567c571ad075551f751444 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-01T02:16:42.543 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:16:42 smithi116 podman[164906]: 2024-06-01 02:16:42.35466805 +0000 UTC m=+0.401791752 container attach 743c8cad2b0c156bd75a2bc3d9f88607192e2c133f567c571ad075551f751444 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-06-01T02:16:42.843 INFO:journalctl@ceph.osd.2.smithi116.stdout:Jun 01 02:16:42 smithi116 podman[164906]: 2024-06-01 02:16:42.560518564 +0000 UTC m=+0.607642264 container died 743c8cad2b0c156bd75a2bc3d9f88607192e2c133f567c571ad075551f751444 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=) 2024-06-01T02:16:43.252 DEBUG:teuthology.orchestra.run.smithi116:> sudo pkill -f 'journalctl -f -n 0 -u ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.2.service' 2024-06-01T02:16:43.293 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-01T02:16:43.293 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-06-01T02:16:43.294 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-06-01T02:16:43.294 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.3 2024-06-01T02:16:43.542 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:16:43 smithi116 systemd[1]: Stopping Ceph osd.3 for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:16:43.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:16:43 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:16:43.649+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:16:44.042 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:16:43 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3[126981]: 2024-06-01T02:16:43.644+0000 7f4c4f7cf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-01T02:16:44.042 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:16:43 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3[126981]: 2024-06-01T02:16:43.644+0000 7f4c4f7cf700 -1 osd.3 196 *** Got signal Terminated *** 2024-06-01T02:16:44.042 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:16:43 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3[126981]: 2024-06-01T02:16:43.644+0000 7f4c4f7cf700 -1 osd.3 196 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-01T02:16:44.676 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:16:44 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:16:44.517+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:16:44.928 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:16:44 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:16:44.679+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:16:45.193 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:16:44 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:16:44.932+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.463978+0000 front 2024-06-01T02:16:22.464056+0000 (oldest deadline 2024-06-01T02:16:44.764055+0000) 2024-06-01T02:16:45.292 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:16:44 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3[126981]: 2024-06-01T02:16:44.892+0000 7f4c4bdc3700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:18.389247+0000 front 2024-06-01T02:16:18.389298+0000 (oldest deadline 2024-06-01T02:16:44.289437+0000) 2024-06-01T02:16:45.931 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:16:45 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:16:45.684+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:16:45.932 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:16:45 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:16:45.484+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:16:46.193 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:16:45 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3[126981]: 2024-06-01T02:16:45.860+0000 7f4c4bdc3700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:18.389247+0000 front 2024-06-01T02:16:18.389298+0000 (oldest deadline 2024-06-01T02:16:44.289437+0000) 2024-06-01T02:16:46.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:16:45 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:16:45.936+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.463978+0000 front 2024-06-01T02:16:22.464056+0000 (oldest deadline 2024-06-01T02:16:44.764055+0000) 2024-06-01T02:16:46.194 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:16:46 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[101715]: 2024-06-01T02:16:46.002+0000 7f003c884700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.717670+0000 front 2024-06-01T02:16:21.717724+0000 (oldest deadline 2024-06-01T02:16:45.217581+0000) 2024-06-01T02:16:46.693 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:16:46 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:16:46.446+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:16:46.989 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:16:46 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:16:46.916+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.463978+0000 front 2024-06-01T02:16:22.464056+0000 (oldest deadline 2024-06-01T02:16:44.764055+0000) 2024-06-01T02:16:46.989 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:16:46 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:16:46.729+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:16:47.131 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:16:46 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3[126981]: 2024-06-01T02:16:46.882+0000 7f4c4bdc3700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:18.389247+0000 front 2024-06-01T02:16:18.389298+0000 (oldest deadline 2024-06-01T02:16:44.289437+0000) 2024-06-01T02:16:47.429 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:16:46 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[101715]: 2024-06-01T02:16:46.993+0000 7f003c884700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.717670+0000 front 2024-06-01T02:16:21.717724+0000 (oldest deadline 2024-06-01T02:16:45.217581+0000) 2024-06-01T02:16:47.681 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:16:47 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:16:47.432+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:16:47.943 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:16:47 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:16:47.886+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.463978+0000 front 2024-06-01T02:16:22.464056+0000 (oldest deadline 2024-06-01T02:16:44.764055+0000) 2024-06-01T02:16:47.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:16:47 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:16:47.684+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:16:48.092 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:16:47 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3[126981]: 2024-06-01T02:16:47.896+0000 7f4c4bdc3700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:18.389247+0000 front 2024-06-01T02:16:18.389298+0000 (oldest deadline 2024-06-01T02:16:44.289437+0000) 2024-06-01T02:16:48.417 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:16:47 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[101715]: 2024-06-01T02:16:47.977+0000 7f003c884700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.717670+0000 front 2024-06-01T02:16:21.717724+0000 (oldest deadline 2024-06-01T02:16:45.217581+0000) 2024-06-01T02:16:48.693 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:16:48 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:16:48.421+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:16:48.694 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:16:48 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:16:48.662+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:16:49.193 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:16:49 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[101715]: 2024-06-01T02:16:49.026+0000 7f003c884700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.717670+0000 front 2024-06-01T02:16:21.717724+0000 (oldest deadline 2024-06-01T02:16:45.217581+0000) 2024-06-01T02:16:49.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:16:48 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:16:48.893+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.463978+0000 front 2024-06-01T02:16:22.464056+0000 (oldest deadline 2024-06-01T02:16:44.764055+0000) 2024-06-01T02:16:49.292 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:16:49 smithi116 podman[165013]: 2024-06-01 02:16:49.212174909 +0000 UTC m=+5.694124002 container died bd00ad2c25566eeec3d5acf3c02e7d6ebf64fc5dd23a587cf8f36bce13deeb8f (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-06-01T02:16:49.792 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:16:49 smithi116 podman[165013]: 2024-06-01 02:16:49.571924331 +0000 UTC m=+6.053873424 container cleanup bd00ad2c25566eeec3d5acf3c02e7d6ebf64fc5dd23a587cf8f36bce13deeb8f (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528) 2024-06-01T02:16:49.792 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:16:49 smithi116 bash[165013]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3 2024-06-01T02:16:49.844 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:16:49 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:16:49.468+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:16:49.844 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:16:49 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:16:49.621+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:16:50.161 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:16:49 smithi116 podman[165148]: 2024-06-01 02:16:49.828433251 +0000 UTC m=+0.636632858 container remove bd00ad2c25566eeec3d5acf3c02e7d6ebf64fc5dd23a587cf8f36bce13deeb8f (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD) 2024-06-01T02:16:50.193 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:16:50 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[101715]: 2024-06-01T02:16:50.052+0000 7f003c884700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.717670+0000 front 2024-06-01T02:16:21.717724+0000 (oldest deadline 2024-06-01T02:16:45.217581+0000) 2024-06-01T02:16:50.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:16:49 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:16:49.848+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.463978+0000 front 2024-06-01T02:16:22.464056+0000 (oldest deadline 2024-06-01T02:16:44.764055+0000) 2024-06-01T02:16:50.792 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:16:50 smithi116 podman[165241]: 2024-06-01 02:16:50.449735917 +0000 UTC m=+0.017181945 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:16:50.793 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:16:50 smithi116 podman[165241]: 2024-06-01 02:16:50.55949034 +0000 UTC m=+0.126936397 container create c27ea1e000edca0eded90b502f4109dd0fc7f57d79deef2f47fff7644aabfc17 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3-deactivate, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-06-01T02:16:50.793 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:16:50 smithi116 podman[165241]: 2024-06-01 02:16:50.787547134 +0000 UTC m=+0.354993160 container init c27ea1e000edca0eded90b502f4109dd0fc7f57d79deef2f47fff7644aabfc17 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-01T02:16:50.793 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:16:50 smithi116 podman[165241]: 2024-06-01 02:16:50.790950238 +0000 UTC m=+0.358396254 container start c27ea1e000edca0eded90b502f4109dd0fc7f57d79deef2f47fff7644aabfc17 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-06-01T02:16:50.864 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:16:50 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:16:50.593+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:16:50.865 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:16:50 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:16:50.458+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:16:51.193 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:16:51 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[101715]: 2024-06-01T02:16:51.049+0000 7f003c884700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.717670+0000 front 2024-06-01T02:16:21.717724+0000 (oldest deadline 2024-06-01T02:16:45.217581+0000) 2024-06-01T02:16:51.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:16:50 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:16:50.869+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.463978+0000 front 2024-06-01T02:16:22.464056+0000 (oldest deadline 2024-06-01T02:16:44.764055+0000) 2024-06-01T02:16:51.292 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:16:50 smithi116 podman[165241]: 2024-06-01 02:16:50.895505621 +0000 UTC m=+0.462951647 container attach c27ea1e000edca0eded90b502f4109dd0fc7f57d79deef2f47fff7644aabfc17 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-01T02:16:51.292 INFO:journalctl@ceph.osd.3.smithi116.stdout:Jun 01 02:16:51 smithi116 podman[165241]: 2024-06-01 02:16:51.073412246 +0000 UTC m=+0.640858258 container died c27ea1e000edca0eded90b502f4109dd0fc7f57d79deef2f47fff7644aabfc17 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528) 2024-06-01T02:16:51.693 DEBUG:teuthology.orchestra.run.smithi116:> sudo pkill -f 'journalctl -f -n 0 -u ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.3.service' 2024-06-01T02:16:51.736 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-01T02:16:51.737 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-06-01T02:16:51.737 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-06-01T02:16:51.737 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.4 2024-06-01T02:16:51.767 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:16:51 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:16:51.459+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:16:51.768 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:16:51 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:16:51.588+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:16:52.037 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:16:51 smithi161 systemd[1]: Stopping Ceph osd.4 for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:16:52.038 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:16:51 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:16:51.859+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.463978+0000 front 2024-06-01T02:16:22.464056+0000 (oldest deadline 2024-06-01T02:16:44.764055+0000) 2024-06-01T02:16:52.421 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:16:52 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[101715]: 2024-06-01T02:16:52.041+0000 7f003c884700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.717670+0000 front 2024-06-01T02:16:21.717724+0000 (oldest deadline 2024-06-01T02:16:45.217581+0000) 2024-06-01T02:16:52.421 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:16:52 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[101715]: 2024-06-01T02:16:52.084+0000 7f0044298700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-01T02:16:52.421 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:16:52 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[101715]: 2024-06-01T02:16:52.084+0000 7f0044298700 -1 osd.4 196 *** Got signal Terminated *** 2024-06-01T02:16:52.421 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:16:52 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[101715]: 2024-06-01T02:16:52.084+0000 7f0044298700 -1 osd.4 196 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-01T02:16:52.693 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:16:52 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:16:52.576+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:16:52.694 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:16:52 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:16:52.425+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:16:53.193 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:16:53 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[101715]: 2024-06-01T02:16:53.077+0000 7f003c884700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.717670+0000 front 2024-06-01T02:16:21.717724+0000 (oldest deadline 2024-06-01T02:16:45.217581+0000) 2024-06-01T02:16:53.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:16:52 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:16:52.880+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.463978+0000 front 2024-06-01T02:16:22.464056+0000 (oldest deadline 2024-06-01T02:16:44.764055+0000) 2024-06-01T02:16:53.580 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:16:53 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:16:53.421+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:16:53.580 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:16:53 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:16:53.421+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:28.399711+0000 front 2024-06-01T02:16:28.399723+0000 (oldest deadline 2024-06-01T02:16:52.499699+0000) 2024-06-01T02:16:53.888 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:16:53 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:16:53.584+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:16:54.193 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:16:54 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[101715]: 2024-06-01T02:16:54.053+0000 7f003c884700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.717670+0000 front 2024-06-01T02:16:21.717724+0000 (oldest deadline 2024-06-01T02:16:45.217581+0000) 2024-06-01T02:16:54.194 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:16:54 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[101715]: 2024-06-01T02:16:54.053+0000 7f003c884700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:30.418943+0000 front 2024-06-01T02:16:30.418888+0000 (oldest deadline 2024-06-01T02:16:53.918746+0000) 2024-06-01T02:16:54.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:16:53 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:16:53.892+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.463978+0000 front 2024-06-01T02:16:22.464056+0000 (oldest deadline 2024-06-01T02:16:44.764055+0000) 2024-06-01T02:16:54.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:16:53 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:16:53.892+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:27.065127+0000 front 2024-06-01T02:16:27.065136+0000 (oldest deadline 2024-06-01T02:16:52.965058+0000) 2024-06-01T02:16:54.936 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:16:54 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:16:54.465+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:16:54.937 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:16:54 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:16:54.465+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:28.399711+0000 front 2024-06-01T02:16:28.399723+0000 (oldest deadline 2024-06-01T02:16:52.499699+0000) 2024-06-01T02:16:54.937 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:16:54 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:16:54.601+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:16:55.193 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:16:55 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[101715]: 2024-06-01T02:16:55.053+0000 7f003c884700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.717670+0000 front 2024-06-01T02:16:21.717724+0000 (oldest deadline 2024-06-01T02:16:45.217581+0000) 2024-06-01T02:16:55.193 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:16:55 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[101715]: 2024-06-01T02:16:55.053+0000 7f003c884700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:30.418943+0000 front 2024-06-01T02:16:30.418888+0000 (oldest deadline 2024-06-01T02:16:53.918746+0000) 2024-06-01T02:16:55.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:16:54 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:16:54.941+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.463978+0000 front 2024-06-01T02:16:22.464056+0000 (oldest deadline 2024-06-01T02:16:44.764055+0000) 2024-06-01T02:16:55.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:16:54 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:16:54.941+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:27.065127+0000 front 2024-06-01T02:16:27.065136+0000 (oldest deadline 2024-06-01T02:16:52.965058+0000) 2024-06-01T02:16:55.918 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:16:55 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:16:55.454+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:16:55.919 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:16:55 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:16:55.454+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:28.399711+0000 front 2024-06-01T02:16:28.399723+0000 (oldest deadline 2024-06-01T02:16:52.499699+0000) 2024-06-01T02:16:55.919 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:16:55 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:16:55.572+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:16:55.919 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:16:55 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:16:55.572+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:29.767566+0000 front 2024-06-01T02:16:29.767739+0000 (oldest deadline 2024-06-01T02:16:55.067675+0000) 2024-06-01T02:16:56.193 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:16:55 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:16:55.923+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.463978+0000 front 2024-06-01T02:16:22.464056+0000 (oldest deadline 2024-06-01T02:16:44.764055+0000) 2024-06-01T02:16:56.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:16:55 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:16:55.923+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:27.065127+0000 front 2024-06-01T02:16:27.065136+0000 (oldest deadline 2024-06-01T02:16:52.965058+0000) 2024-06-01T02:16:56.194 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:16:56 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[101715]: 2024-06-01T02:16:56.091+0000 7f003c884700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.717670+0000 front 2024-06-01T02:16:21.717724+0000 (oldest deadline 2024-06-01T02:16:45.217581+0000) 2024-06-01T02:16:56.194 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:16:56 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[101715]: 2024-06-01T02:16:56.091+0000 7f003c884700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:30.418943+0000 front 2024-06-01T02:16:30.418888+0000 (oldest deadline 2024-06-01T02:16:53.918746+0000) 2024-06-01T02:16:56.693 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:16:56 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:16:56.433+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:16:56.693 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:16:56 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:16:56.433+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:28.399711+0000 front 2024-06-01T02:16:28.399723+0000 (oldest deadline 2024-06-01T02:16:52.499699+0000) 2024-06-01T02:16:56.694 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:16:56 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:16:56.595+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:16:56.694 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:16:56 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:16:56.595+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:29.767566+0000 front 2024-06-01T02:16:29.767739+0000 (oldest deadline 2024-06-01T02:16:55.067675+0000) 2024-06-01T02:16:57.193 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:16:56 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:16:56.887+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.463978+0000 front 2024-06-01T02:16:22.464056+0000 (oldest deadline 2024-06-01T02:16:44.764055+0000) 2024-06-01T02:16:57.193 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:16:56 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:16:56.887+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:27.065127+0000 front 2024-06-01T02:16:27.065136+0000 (oldest deadline 2024-06-01T02:16:52.965058+0000) 2024-06-01T02:16:57.194 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:16:57 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[101715]: 2024-06-01T02:16:57.061+0000 7f003c884700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.717670+0000 front 2024-06-01T02:16:21.717724+0000 (oldest deadline 2024-06-01T02:16:45.217581+0000) 2024-06-01T02:16:57.194 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:16:57 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4[101715]: 2024-06-01T02:16:57.061+0000 7f003c884700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:30.418943+0000 front 2024-06-01T02:16:30.418888+0000 (oldest deadline 2024-06-01T02:16:53.918746+0000) 2024-06-01T02:16:57.773 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:16:57 smithi161 podman[149136]: 2024-06-01 02:16:57.487957284 +0000 UTC m=+5.533802905 container stop 0084b9edf7a05ef25aaa4c197befae6f687658690e7d27c1323933da3c744ca6 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-06-01T02:16:57.773 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:16:57 smithi161 podman[149136]: 2024-06-01 02:16:57.513674925 +0000 UTC m=+5.559520551 container died 0084b9edf7a05ef25aaa4c197befae6f687658690e7d27c1323933da3c744ca6 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528) 2024-06-01T02:16:57.773 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:16:57 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:16:57.624+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:16:57.774 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:16:57 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:16:57.624+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:29.767566+0000 front 2024-06-01T02:16:29.767739+0000 (oldest deadline 2024-06-01T02:16:55.067675+0000) 2024-06-01T02:16:57.774 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:16:57 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:16:57.481+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:16:57.774 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:16:57 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:16:57.481+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:28.399711+0000 front 2024-06-01T02:16:28.399723+0000 (oldest deadline 2024-06-01T02:16:52.499699+0000) 2024-06-01T02:16:58.193 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:16:57 smithi161 podman[149136]: 2024-06-01 02:16:57.952246699 +0000 UTC m=+5.998092356 container cleanup 0084b9edf7a05ef25aaa4c197befae6f687658690e7d27c1323933da3c744ca6 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-01T02:16:58.194 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:16:57 smithi161 bash[149136]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4 2024-06-01T02:16:58.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:16:57 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:16:57.910+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.463978+0000 front 2024-06-01T02:16:22.464056+0000 (oldest deadline 2024-06-01T02:16:44.764055+0000) 2024-06-01T02:16:58.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:16:57 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:16:57.910+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:27.065127+0000 front 2024-06-01T02:16:27.065136+0000 (oldest deadline 2024-06-01T02:16:52.965058+0000) 2024-06-01T02:16:58.588 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:16:58 smithi161 podman[149154]: 2024-06-01 02:16:58.329435464 +0000 UTC m=+0.830398219 container remove 0084b9edf7a05ef25aaa4c197befae6f687658690e7d27c1323933da3c744ca6 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True) 2024-06-01T02:16:58.588 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:16:58 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:16:58.504+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:16:58.588 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:16:58 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:16:58.504+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:28.399711+0000 front 2024-06-01T02:16:28.399723+0000 (oldest deadline 2024-06-01T02:16:52.499699+0000) 2024-06-01T02:16:58.874 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:16:58 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:16:58.638+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:16:58.874 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:16:58 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:16:58.638+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:29.767566+0000 front 2024-06-01T02:16:29.767739+0000 (oldest deadline 2024-06-01T02:16:55.067675+0000) 2024-06-01T02:16:58.875 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:16:58 smithi161 podman[149364]: 2024-06-01 02:16:58.67307749 +0000 UTC m=+0.018189152 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:16:58.875 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:16:58 smithi161 podman[149364]: 2024-06-01 02:16:58.771835754 +0000 UTC m=+0.116947402 container create a294847753755516a328d0a593831133d9644557db10a05e8a6809e16c436a5b (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-06-01T02:16:59.171 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:16:58 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:16:58.930+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.463978+0000 front 2024-06-01T02:16:22.464056+0000 (oldest deadline 2024-06-01T02:16:44.764055+0000) 2024-06-01T02:16:59.172 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:16:58 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:16:58.930+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:27.065127+0000 front 2024-06-01T02:16:27.065136+0000 (oldest deadline 2024-06-01T02:16:52.965058+0000) 2024-06-01T02:16:59.172 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:16:58 smithi161 podman[149364]: 2024-06-01 02:16:58.983118682 +0000 UTC m=+0.328230339 container init a294847753755516a328d0a593831133d9644557db10a05e8a6809e16c436a5b (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc) 2024-06-01T02:16:59.172 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:16:58 smithi161 podman[149364]: 2024-06-01 02:16:58.986662896 +0000 UTC m=+0.331774551 container start a294847753755516a328d0a593831133d9644557db10a05e8a6809e16c436a5b (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc) 2024-06-01T02:16:59.172 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:16:59 smithi161 podman[149364]: 2024-06-01 02:16:59.06591559 +0000 UTC m=+0.411027251 container attach a294847753755516a328d0a593831133d9644557db10a05e8a6809e16c436a5b (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, ceph=True) 2024-06-01T02:16:59.446 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jun 01 02:16:59 smithi161 podman[149364]: 2024-06-01 02:16:59.304024024 +0000 UTC m=+0.649135681 container died a294847753755516a328d0a593831133d9644557db10a05e8a6809e16c436a5b (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-06-01T02:16:59.831 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:16:59 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:16:59.455+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:16:59.831 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:16:59 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:16:59.455+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:28.399711+0000 front 2024-06-01T02:16:28.399723+0000 (oldest deadline 2024-06-01T02:16:52.499699+0000) 2024-06-01T02:16:59.832 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:16:59 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:16:59.627+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:16:59.832 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:16:59 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:16:59.627+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:29.767566+0000 front 2024-06-01T02:16:29.767739+0000 (oldest deadline 2024-06-01T02:16:55.067675+0000) 2024-06-01T02:16:59.855 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.4.service' 2024-06-01T02:16:59.936 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-01T02:16:59.936 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-06-01T02:16:59.936 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-06-01T02:16:59.936 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.5 2024-06-01T02:17:00.194 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:00 smithi161 systemd[1]: Stopping Ceph osd.5 for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:17:00.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:16:59 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:16:59.911+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.463978+0000 front 2024-06-01T02:16:22.464056+0000 (oldest deadline 2024-06-01T02:16:44.764055+0000) 2024-06-01T02:17:00.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:16:59 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:16:59.911+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:27.065127+0000 front 2024-06-01T02:16:27.065136+0000 (oldest deadline 2024-06-01T02:16:52.965058+0000) 2024-06-01T02:17:00.630 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:00 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:17:00.313+0000 7fecafbde700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-01T02:17:00.630 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:00 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:17:00.313+0000 7fecafbde700 -1 osd.5 196 *** Got signal Terminated *** 2024-06-01T02:17:00.630 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:00 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:17:00.313+0000 7fecafbde700 -1 osd.5 196 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-01T02:17:00.631 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:00 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:17:00.431+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:17:00.631 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:00 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:17:00.431+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:28.399711+0000 front 2024-06-01T02:16:28.399723+0000 (oldest deadline 2024-06-01T02:16:52.499699+0000) 2024-06-01T02:17:00.906 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:00 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:00.634+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:17:00.906 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:00 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:00.634+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:29.767566+0000 front 2024-06-01T02:16:29.767739+0000 (oldest deadline 2024-06-01T02:16:55.067675+0000) 2024-06-01T02:17:00.906 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:00 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:00.634+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:35.068180+0000 front 2024-06-01T02:16:35.068257+0000 (oldest deadline 2024-06-01T02:16:59.768106+0000) 2024-06-01T02:17:01.193 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:00 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:00.910+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.463978+0000 front 2024-06-01T02:16:22.464056+0000 (oldest deadline 2024-06-01T02:16:44.764055+0000) 2024-06-01T02:17:01.193 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:00 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:00.910+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:27.065127+0000 front 2024-06-01T02:16:27.065136+0000 (oldest deadline 2024-06-01T02:16:52.965058+0000) 2024-06-01T02:17:01.803 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:01 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:17:01.472+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:17:01.803 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:01 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:17:01.472+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:28.399711+0000 front 2024-06-01T02:16:28.399723+0000 (oldest deadline 2024-06-01T02:16:52.499699+0000) 2024-06-01T02:17:01.803 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:01 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:01.587+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:17:01.804 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:01 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:01.587+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:29.767566+0000 front 2024-06-01T02:16:29.767739+0000 (oldest deadline 2024-06-01T02:16:55.067675+0000) 2024-06-01T02:17:01.804 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:01 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:01.587+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:35.068180+0000 front 2024-06-01T02:16:35.068257+0000 (oldest deadline 2024-06-01T02:16:59.768106+0000) 2024-06-01T02:17:02.193 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:01 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:01.867+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.463978+0000 front 2024-06-01T02:16:22.464056+0000 (oldest deadline 2024-06-01T02:16:44.764055+0000) 2024-06-01T02:17:02.193 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:01 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:01.867+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:27.065127+0000 front 2024-06-01T02:16:27.065136+0000 (oldest deadline 2024-06-01T02:16:52.965058+0000) 2024-06-01T02:17:02.193 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:01 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:01.867+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:38.866005+0000 front 2024-06-01T02:16:38.865958+0000 (oldest deadline 2024-06-01T02:17:01.765902+0000) 2024-06-01T02:17:02.550 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:02 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:17:02.454+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:17:02.550 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:02 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:17:02.454+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:28.399711+0000 front 2024-06-01T02:16:28.399723+0000 (oldest deadline 2024-06-01T02:16:52.499699+0000) 2024-06-01T02:17:02.550 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:02 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:17:02.454+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:33.600687+0000 front 2024-06-01T02:16:33.600824+0000 (oldest deadline 2024-06-01T02:17:01.801096+0000) 2024-06-01T02:17:02.903 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:02 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:02.574+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:17:02.903 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:02 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:02.574+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:29.767566+0000 front 2024-06-01T02:16:29.767739+0000 (oldest deadline 2024-06-01T02:16:55.067675+0000) 2024-06-01T02:17:02.903 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:02 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:02.574+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:35.068180+0000 front 2024-06-01T02:16:35.068257+0000 (oldest deadline 2024-06-01T02:16:59.768106+0000) 2024-06-01T02:17:03.193 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:02 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:02.908+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.463978+0000 front 2024-06-01T02:16:22.464056+0000 (oldest deadline 2024-06-01T02:16:44.764055+0000) 2024-06-01T02:17:03.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:02 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:02.908+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:27.065127+0000 front 2024-06-01T02:16:27.065136+0000 (oldest deadline 2024-06-01T02:16:52.965058+0000) 2024-06-01T02:17:03.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:02 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:02.908+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:38.866005+0000 front 2024-06-01T02:16:38.865958+0000 (oldest deadline 2024-06-01T02:17:01.765902+0000) 2024-06-01T02:17:03.539 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:03 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:17:03.436+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:17:03.540 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:03 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:17:03.436+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:28.399711+0000 front 2024-06-01T02:16:28.399723+0000 (oldest deadline 2024-06-01T02:16:52.499699+0000) 2024-06-01T02:17:03.540 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:03 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:17:03.436+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:33.600687+0000 front 2024-06-01T02:16:33.600824+0000 (oldest deadline 2024-06-01T02:17:01.801096+0000) 2024-06-01T02:17:03.793 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:03 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:03.587+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:17:03.793 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:03 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:03.587+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:29.767566+0000 front 2024-06-01T02:16:29.767739+0000 (oldest deadline 2024-06-01T02:16:55.067675+0000) 2024-06-01T02:17:03.793 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:03 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:03.587+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:35.068180+0000 front 2024-06-01T02:16:35.068257+0000 (oldest deadline 2024-06-01T02:16:59.768106+0000) 2024-06-01T02:17:04.133 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:03 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:03.905+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.463978+0000 front 2024-06-01T02:16:22.464056+0000 (oldest deadline 2024-06-01T02:16:44.764055+0000) 2024-06-01T02:17:04.134 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:03 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:03.906+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:27.065127+0000 front 2024-06-01T02:16:27.065136+0000 (oldest deadline 2024-06-01T02:16:52.965058+0000) 2024-06-01T02:17:04.134 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:03 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:03.906+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:38.866005+0000 front 2024-06-01T02:16:38.865958+0000 (oldest deadline 2024-06-01T02:17:01.765902+0000) 2024-06-01T02:17:04.662 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:04.634+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:17:04.663 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:04.634+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:29.767566+0000 front 2024-06-01T02:16:29.767739+0000 (oldest deadline 2024-06-01T02:16:55.067675+0000) 2024-06-01T02:17:04.663 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:04.634+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:35.068180+0000 front 2024-06-01T02:16:35.068257+0000 (oldest deadline 2024-06-01T02:16:59.768106+0000) 2024-06-01T02:17:04.663 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:17:04.402+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:17:04.663 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:17:04.402+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:28.399711+0000 front 2024-06-01T02:16:28.399723+0000 (oldest deadline 2024-06-01T02:16:52.499699+0000) 2024-06-01T02:17:04.663 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:17:04.402+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:33.600687+0000 front 2024-06-01T02:16:33.600824+0000 (oldest deadline 2024-06-01T02:17:01.801096+0000) 2024-06-01T02:17:05.193 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:04.931+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.463978+0000 front 2024-06-01T02:16:22.464056+0000 (oldest deadline 2024-06-01T02:16:44.764055+0000) 2024-06-01T02:17:05.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:04.931+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:27.065127+0000 front 2024-06-01T02:16:27.065136+0000 (oldest deadline 2024-06-01T02:16:52.965058+0000) 2024-06-01T02:17:05.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:04 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:04.931+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:38.866005+0000 front 2024-06-01T02:16:38.865958+0000 (oldest deadline 2024-06-01T02:17:01.765902+0000) 2024-06-01T02:17:05.586 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: *** Caught signal (Segmentation fault) ** 2024-06-01T02:17:05.586 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: in thread 7fec9b1b0700 thread_name:safe_timer 2024-06-01T02:17:05.586 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-01T02:17:05.586 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fecb7035d20] 2024-06-01T02:17:05.587 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2: (BlueStore::_txc_create(BlueStore::Collection*, BlueStore::OpSequencer*, std::__cxx11::list >*, boost::intrusive_ptr)+0x40a) [0x56436741825a] 2024-06-01T02:17:05.587 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 3: (BlueStore::queue_transactions(boost::intrusive_ptr&, std::vector >&, boost::intrusive_ptr, ThreadPool::TPHandle*)+0x1f8) [0x5643674a5db8] 2024-06-01T02:17:05.587 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 4: (non-virtual thunk to PrimaryLogPG::queue_transactions(std::vector >&, boost::intrusive_ptr)+0x53) [0x564367091343] 2024-06-01T02:17:05.587 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 5: (ReplicatedBackend::submit_transaction(hobject_t const&, object_stat_sum_t const&, eversion_t const&, std::unique_ptr >&&, eversion_t const&, eversion_t const&, std::vector >&&, std::optional&, Context*, unsigned long, osd_reqid_t, boost::intrusive_ptr)+0x7b1) [0x564367316a61] 2024-06-01T02:17:05.587 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 6: (PrimaryLogPG::issue_repop(PrimaryLogPG::RepGather*, PrimaryLogPG::OpContext*)+0x424) [0x56436700d074] 2024-06-01T02:17:05.587 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 7: (PrimaryLogPG::simple_opc_submit(std::unique_ptr >)+0x5a) [0x56436700f0da] 2024-06-01T02:17:05.587 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 8: (PrimaryLogPG::handle_watch_timeout(std::shared_ptr)+0x873) [0x564367011043] 2024-06-01T02:17:05.587 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 9: (HandleWatchTimeout::complete(int)+0x11a) [0x564366f7252a] 2024-06-01T02:17:05.587 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 10: (CommonSafeTimer::timer_thread()+0x155) [0x5643675c7b15] 2024-06-01T02:17:05.587 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 11: (CommonSafeTimerThread::entry()+0x11) [0x5643675c9341] 2024-06-01T02:17:05.587 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fecb702b1ca] 2024-06-01T02:17:05.587 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 13: clone() 2024-06-01T02:17:05.588 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:17:05.482+0000 7fec9b1b0700 -1 *** Caught signal (Segmentation fault) ** 2024-06-01T02:17:05.588 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: in thread 7fec9b1b0700 thread_name:safe_timer 2024-06-01T02:17:05.588 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:17:05.588 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-01T02:17:05.588 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fecb7035d20] 2024-06-01T02:17:05.588 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2: (BlueStore::_txc_create(BlueStore::Collection*, BlueStore::OpSequencer*, std::__cxx11::list >*, boost::intrusive_ptr)+0x40a) [0x56436741825a] 2024-06-01T02:17:05.588 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 3: (BlueStore::queue_transactions(boost::intrusive_ptr&, std::vector >&, boost::intrusive_ptr, ThreadPool::TPHandle*)+0x1f8) [0x5643674a5db8] 2024-06-01T02:17:05.588 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 4: (non-virtual thunk to PrimaryLogPG::queue_transactions(std::vector >&, boost::intrusive_ptr)+0x53) [0x564367091343] 2024-06-01T02:17:05.588 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 5: (ReplicatedBackend::submit_transaction(hobject_t const&, object_stat_sum_t const&, eversion_t const&, std::unique_ptr >&&, eversion_t const&, eversion_t const&, std::vector >&&, std::optional&, Context*, unsigned long, osd_reqid_t, boost::intrusive_ptr)+0x7b1) [0x564367316a61] 2024-06-01T02:17:05.588 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 6: (PrimaryLogPG::issue_repop(PrimaryLogPG::RepGather*, PrimaryLogPG::OpContext*)+0x424) [0x56436700d074] 2024-06-01T02:17:05.588 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 7: (PrimaryLogPG::simple_opc_submit(std::unique_ptr >)+0x5a) [0x56436700f0da] 2024-06-01T02:17:05.588 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 8: (PrimaryLogPG::handle_watch_timeout(std::shared_ptr)+0x873) [0x564367011043] 2024-06-01T02:17:05.588 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 9: (HandleWatchTimeout::complete(int)+0x11a) [0x564366f7252a] 2024-06-01T02:17:05.589 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 10: (CommonSafeTimer::timer_thread()+0x155) [0x5643675c7b15] 2024-06-01T02:17:05.589 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 11: (CommonSafeTimerThread::entry()+0x11) [0x5643675c9341] 2024-06-01T02:17:05.589 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fecb702b1ca] 2024-06-01T02:17:05.589 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 13: clone() 2024-06-01T02:17:05.589 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-06-01T02:17:05.589 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:17:05.589 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -6058> 2024-06-01T02:16:44.517+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:17:05.589 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -5696> 2024-06-01T02:16:45.484+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:17:05.589 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -5626> 2024-06-01T02:16:46.446+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:17:05.589 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -5009> 2024-06-01T02:16:47.432+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:17:05.590 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -4227> 2024-06-01T02:16:48.421+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:17:05.590 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -4067> 2024-06-01T02:16:49.468+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:17:05.590 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -3892> 2024-06-01T02:16:50.458+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:17:05.590 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -3758> 2024-06-01T02:16:51.459+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:17:05.590 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -3632> 2024-06-01T02:16:52.425+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:17:05.590 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -3545> 2024-06-01T02:16:53.421+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:17:05.590 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -3544> 2024-06-01T02:16:53.421+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:28.399711+0000 front 2024-06-01T02:16:28.399723+0000 (oldest deadline 2024-06-01T02:16:52.499699+0000) 2024-06-01T02:17:05.591 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -3428> 2024-06-01T02:16:54.465+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:17:05.591 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -3427> 2024-06-01T02:16:54.465+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:28.399711+0000 front 2024-06-01T02:16:28.399723+0000 (oldest deadline 2024-06-01T02:16:52.499699+0000) 2024-06-01T02:17:05.591 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -2819> 2024-06-01T02:16:55.454+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:17:05.591 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -2818> 2024-06-01T02:16:55.454+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:28.399711+0000 front 2024-06-01T02:16:28.399723+0000 (oldest deadline 2024-06-01T02:16:52.499699+0000) 2024-06-01T02:17:05.591 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -2127> 2024-06-01T02:16:56.433+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:17:05.591 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -2126> 2024-06-01T02:16:56.433+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:28.399711+0000 front 2024-06-01T02:16:28.399723+0000 (oldest deadline 2024-06-01T02:16:52.499699+0000) 2024-06-01T02:17:05.591 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -2041> 2024-06-01T02:16:57.481+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:17:05.591 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -2040> 2024-06-01T02:16:57.481+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:28.399711+0000 front 2024-06-01T02:16:28.399723+0000 (oldest deadline 2024-06-01T02:16:52.499699+0000) 2024-06-01T02:17:05.591 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -1939> 2024-06-01T02:16:58.504+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:17:05.591 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -1938> 2024-06-01T02:16:58.504+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:28.399711+0000 front 2024-06-01T02:16:28.399723+0000 (oldest deadline 2024-06-01T02:16:52.499699+0000) 2024-06-01T02:17:05.591 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -1816> 2024-06-01T02:16:59.455+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:17:05.591 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -1815> 2024-06-01T02:16:59.455+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:28.399711+0000 front 2024-06-01T02:16:28.399723+0000 (oldest deadline 2024-06-01T02:16:52.499699+0000) 2024-06-01T02:17:05.591 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -1647> 2024-06-01T02:17:00.313+0000 7fecafbde700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-01T02:17:05.592 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -1646> 2024-06-01T02:17:00.313+0000 7fecafbde700 -1 osd.5 196 *** Got signal Terminated *** 2024-06-01T02:17:05.592 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -1644> 2024-06-01T02:17:00.313+0000 7fecafbde700 -1 osd.5 196 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-01T02:17:05.592 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -1641> 2024-06-01T02:17:00.431+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:17:05.592 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -1640> 2024-06-01T02:17:00.431+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:28.399711+0000 front 2024-06-01T02:16:28.399723+0000 (oldest deadline 2024-06-01T02:16:52.499699+0000) 2024-06-01T02:17:05.592 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -1475> 2024-06-01T02:17:01.472+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:17:05.592 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -1474> 2024-06-01T02:17:01.472+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:28.399711+0000 front 2024-06-01T02:16:28.399723+0000 (oldest deadline 2024-06-01T02:16:52.499699+0000) 2024-06-01T02:17:05.592 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -1430> 2024-06-01T02:17:02.454+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:17:05.592 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -1429> 2024-06-01T02:17:02.454+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:28.399711+0000 front 2024-06-01T02:16:28.399723+0000 (oldest deadline 2024-06-01T02:16:52.499699+0000) 2024-06-01T02:17:05.592 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -1428> 2024-06-01T02:17:02.454+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:33.600687+0000 front 2024-06-01T02:16:33.600824+0000 (oldest deadline 2024-06-01T02:17:01.801096+0000) 2024-06-01T02:17:05.592 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -1049> 2024-06-01T02:17:03.436+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:17:05.592 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -1048> 2024-06-01T02:17:03.436+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:28.399711+0000 front 2024-06-01T02:16:28.399723+0000 (oldest deadline 2024-06-01T02:16:52.499699+0000) 2024-06-01T02:17:05.592 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -1047> 2024-06-01T02:17:03.436+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:33.600687+0000 front 2024-06-01T02:16:33.600824+0000 (oldest deadline 2024-06-01T02:17:01.801096+0000) 2024-06-01T02:17:05.592 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -442> 2024-06-01T02:17:04.402+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:21.998890+0000 front 2024-06-01T02:16:21.998748+0000 (oldest deadline 2024-06-01T02:16:43.698801+0000) 2024-06-01T02:17:05.593 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -441> 2024-06-01T02:17:04.402+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:28.399711+0000 front 2024-06-01T02:16:28.399723+0000 (oldest deadline 2024-06-01T02:16:52.499699+0000) 2024-06-01T02:17:05.593 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: -440> 2024-06-01T02:17:04.402+0000 7fecac1d2700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:33.600687+0000 front 2024-06-01T02:16:33.600824+0000 (oldest deadline 2024-06-01T02:17:01.801096+0000) 2024-06-01T02:17:05.593 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 0> 2024-06-01T02:17:05.482+0000 7fec9b1b0700 -1 *** Caught signal (Segmentation fault) ** 2024-06-01T02:17:05.593 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: in thread 7fec9b1b0700 thread_name:safe_timer 2024-06-01T02:17:05.593 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:17:05.593 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-01T02:17:05.593 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fecb7035d20] 2024-06-01T02:17:05.593 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2: (BlueStore::_txc_create(BlueStore::Collection*, BlueStore::OpSequencer*, std::__cxx11::list >*, boost::intrusive_ptr)+0x40a) [0x56436741825a] 2024-06-01T02:17:05.593 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 3: (BlueStore::queue_transactions(boost::intrusive_ptr&, std::vector >&, boost::intrusive_ptr, ThreadPool::TPHandle*)+0x1f8) [0x5643674a5db8] 2024-06-01T02:17:05.593 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 4: (non-virtual thunk to PrimaryLogPG::queue_transactions(std::vector >&, boost::intrusive_ptr)+0x53) [0x564367091343] 2024-06-01T02:17:05.593 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 5: (ReplicatedBackend::submit_transaction(hobject_t const&, object_stat_sum_t const&, eversion_t const&, std::unique_ptr >&&, eversion_t const&, eversion_t const&, std::vector >&&, std::optional&, Context*, unsigned long, osd_reqid_t, boost::intrusive_ptr)+0x7b1) [0x564367316a61] 2024-06-01T02:17:05.593 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 6: (PrimaryLogPG::issue_repop(PrimaryLogPG::RepGather*, PrimaryLogPG::OpContext*)+0x424) [0x56436700d074] 2024-06-01T02:17:05.594 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 7: (PrimaryLogPG::simple_opc_submit(std::unique_ptr >)+0x5a) [0x56436700f0da] 2024-06-01T02:17:05.594 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 8: (PrimaryLogPG::handle_watch_timeout(std::shared_ptr)+0x873) [0x564367011043] 2024-06-01T02:17:05.594 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 9: (HandleWatchTimeout::complete(int)+0x11a) [0x564366f7252a] 2024-06-01T02:17:05.594 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 10: (CommonSafeTimer::timer_thread()+0x155) [0x5643675c7b15] 2024-06-01T02:17:05.594 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 11: (CommonSafeTimerThread::entry()+0x11) [0x5643675c9341] 2024-06-01T02:17:05.594 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fecb702b1ca] 2024-06-01T02:17:05.594 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 13: clone() 2024-06-01T02:17:05.594 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-06-01T02:17:05.594 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5[114650]: 2024-06-01T02:17:05.879 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 podman[149557]: 2024-06-01 02:17:05.643864617 +0000 UTC m=+5.455202503 container died b3849cdeb707262a58ce5c90479519664aa4080aadb8283e99ca0277b57f68eb (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5, ceph=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-01T02:17:05.880 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:05.590+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:17:05.880 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:05.590+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:29.767566+0000 front 2024-06-01T02:16:29.767739+0000 (oldest deadline 2024-06-01T02:16:55.067675+0000) 2024-06-01T02:17:05.880 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:05.590+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:35.068180+0000 front 2024-06-01T02:16:35.068257+0000 (oldest deadline 2024-06-01T02:16:59.768106+0000) 2024-06-01T02:17:06.131 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 podman[149557]: 2024-06-01 02:17:05.976470686 +0000 UTC m=+5.787808506 container cleanup b3849cdeb707262a58ce5c90479519664aa4080aadb8283e99ca0277b57f68eb (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-06-01T02:17:06.131 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:05 smithi161 bash[149557]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5 2024-06-01T02:17:06.131 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:05.884+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.463978+0000 front 2024-06-01T02:16:22.464056+0000 (oldest deadline 2024-06-01T02:16:44.764055+0000) 2024-06-01T02:17:06.131 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:05.884+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:27.065127+0000 front 2024-06-01T02:16:27.065136+0000 (oldest deadline 2024-06-01T02:16:52.965058+0000) 2024-06-01T02:17:06.131 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:05 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:05.884+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:38.866005+0000 front 2024-06-01T02:16:38.865958+0000 (oldest deadline 2024-06-01T02:17:01.765902+0000) 2024-06-01T02:17:06.384 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:06 smithi161 podman[150075]: 2024-06-01 02:17:06.136119283 +0000 UTC m=+0.571177827 container remove b3849cdeb707262a58ce5c90479519664aa4080aadb8283e99ca0277b57f68eb (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528) 2024-06-01T02:17:06.384 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:06 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.5.service: Main process exited, code=exited, status=139/n/a 2024-06-01T02:17:06.638 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:06 smithi161 podman[150214]: 2024-06-01 02:17:06.545461639 +0000 UTC m=+0.105669993 container create e658741edbf1cd2b33870d83c41b0d86330d6112a2c35beddacd4f485c245b69 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, ceph=True, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528) 2024-06-01T02:17:06.638 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:06 smithi161 podman[150214]: 2024-06-01 02:17:06.456740395 +0000 UTC m=+0.016948769 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:17:06.638 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:06.571+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:17:06.638 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:06.571+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:29.767566+0000 front 2024-06-01T02:16:29.767739+0000 (oldest deadline 2024-06-01T02:16:55.067675+0000) 2024-06-01T02:17:06.639 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:06.571+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:35.068180+0000 front 2024-06-01T02:16:35.068257+0000 (oldest deadline 2024-06-01T02:16:59.768106+0000) 2024-06-01T02:17:06.911 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:06 smithi161 podman[150214]: 2024-06-01 02:17:06.76441834 +0000 UTC m=+0.324626694 container init e658741edbf1cd2b33870d83c41b0d86330d6112a2c35beddacd4f485c245b69 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-01T02:17:06.912 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:06 smithi161 podman[150214]: 2024-06-01 02:17:06.767870748 +0000 UTC m=+0.328079103 container start e658741edbf1cd2b33870d83c41b0d86330d6112a2c35beddacd4f485c245b69 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS) 2024-06-01T02:17:06.912 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:06 smithi161 podman[150214]: 2024-06-01 02:17:06.847803096 +0000 UTC m=+0.408011496 container attach e658741edbf1cd2b33870d83c41b0d86330d6112a2c35beddacd4f485c245b69 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-06-01T02:17:07.193 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jun 01 02:17:07 smithi161 podman[150214]: 2024-06-01 02:17:07.065654334 +0000 UTC m=+0.625862692 container died e658741edbf1cd2b33870d83c41b0d86330d6112a2c35beddacd4f485c245b69 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-06-01T02:17:07.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:06.916+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.463978+0000 front 2024-06-01T02:16:22.464056+0000 (oldest deadline 2024-06-01T02:16:44.764055+0000) 2024-06-01T02:17:07.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:06.916+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:27.065127+0000 front 2024-06-01T02:16:27.065136+0000 (oldest deadline 2024-06-01T02:16:52.965058+0000) 2024-06-01T02:17:07.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:06.916+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:38.866005+0000 front 2024-06-01T02:16:38.865958+0000 (oldest deadline 2024-06-01T02:17:01.765902+0000) 2024-06-01T02:17:07.604 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:07.576+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:17:07.604 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:07.576+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:29.767566+0000 front 2024-06-01T02:16:29.767739+0000 (oldest deadline 2024-06-01T02:16:55.067675+0000) 2024-06-01T02:17:07.604 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:07.576+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:35.068180+0000 front 2024-06-01T02:16:35.068257+0000 (oldest deadline 2024-06-01T02:16:59.768106+0000) 2024-06-01T02:17:07.624 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.5.service' 2024-06-01T02:17:07.663 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-01T02:17:07.664 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-06-01T02:17:07.664 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-06-01T02:17:07.664 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.6 2024-06-01T02:17:07.938 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:07 smithi161 systemd[1]: Stopping Ceph osd.6 for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:17:08.193 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:07.942+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.463978+0000 front 2024-06-01T02:16:22.464056+0000 (oldest deadline 2024-06-01T02:16:44.764055+0000) 2024-06-01T02:17:08.193 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:07.943+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:27.065127+0000 front 2024-06-01T02:16:27.065136+0000 (oldest deadline 2024-06-01T02:16:52.965058+0000) 2024-06-01T02:17:08.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:07.943+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:38.866005+0000 front 2024-06-01T02:16:38.865958+0000 (oldest deadline 2024-06-01T02:17:01.765902+0000) 2024-06-01T02:17:08.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:07.996+0000 7f7338ec9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-01T02:17:08.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:07.996+0000 7f7338ec9700 -1 osd.6 196 *** Got signal Terminated *** 2024-06-01T02:17:08.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:07 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:07.996+0000 7f7338ec9700 -1 osd.6 196 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-01T02:17:08.943 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:08 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:08.536+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:17:08.943 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:08 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:08.536+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:29.767566+0000 front 2024-06-01T02:16:29.767739+0000 (oldest deadline 2024-06-01T02:16:55.067675+0000) 2024-06-01T02:17:08.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:08 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:08.536+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:35.068180+0000 front 2024-06-01T02:16:35.068257+0000 (oldest deadline 2024-06-01T02:16:59.768106+0000) 2024-06-01T02:17:09.443 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:08 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:08.950+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.463978+0000 front 2024-06-01T02:16:22.464056+0000 (oldest deadline 2024-06-01T02:16:44.764055+0000) 2024-06-01T02:17:09.444 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:08 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:08.950+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:27.065127+0000 front 2024-06-01T02:16:27.065136+0000 (oldest deadline 2024-06-01T02:16:52.965058+0000) 2024-06-01T02:17:09.444 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:08 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:08.950+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:38.866005+0000 front 2024-06-01T02:16:38.865958+0000 (oldest deadline 2024-06-01T02:17:01.765902+0000) 2024-06-01T02:17:09.906 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:09 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:09.568+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:17:09.906 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:09 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:09.568+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:29.767566+0000 front 2024-06-01T02:16:29.767739+0000 (oldest deadline 2024-06-01T02:16:55.067675+0000) 2024-06-01T02:17:09.906 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:09 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:09.568+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:35.068180+0000 front 2024-06-01T02:16:35.068257+0000 (oldest deadline 2024-06-01T02:16:59.768106+0000) 2024-06-01T02:17:10.193 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:09 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:09.910+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.463978+0000 front 2024-06-01T02:16:22.464056+0000 (oldest deadline 2024-06-01T02:16:44.764055+0000) 2024-06-01T02:17:10.193 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:09 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:09.910+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:27.065127+0000 front 2024-06-01T02:16:27.065136+0000 (oldest deadline 2024-06-01T02:16:52.965058+0000) 2024-06-01T02:17:10.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:09 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:09.910+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:38.866005+0000 front 2024-06-01T02:16:38.865958+0000 (oldest deadline 2024-06-01T02:17:01.765902+0000) 2024-06-01T02:17:10.879 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:10 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:10.570+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:17:10.879 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:10 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:10.570+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:29.767566+0000 front 2024-06-01T02:16:29.767739+0000 (oldest deadline 2024-06-01T02:16:55.067675+0000) 2024-06-01T02:17:10.879 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:10 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:10.570+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:35.068180+0000 front 2024-06-01T02:16:35.068257+0000 (oldest deadline 2024-06-01T02:16:59.768106+0000) 2024-06-01T02:17:11.193 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:10 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:10.884+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.463978+0000 front 2024-06-01T02:16:22.464056+0000 (oldest deadline 2024-06-01T02:16:44.764055+0000) 2024-06-01T02:17:11.193 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:10 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:10.884+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:27.065127+0000 front 2024-06-01T02:16:27.065136+0000 (oldest deadline 2024-06-01T02:16:52.965058+0000) 2024-06-01T02:17:11.193 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:10 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:10.884+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:38.866005+0000 front 2024-06-01T02:16:38.865958+0000 (oldest deadline 2024-06-01T02:17:01.765902+0000) 2024-06-01T02:17:11.839 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:11 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:11.540+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:17:11.839 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:11 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:11.540+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:29.767566+0000 front 2024-06-01T02:16:29.767739+0000 (oldest deadline 2024-06-01T02:16:55.067675+0000) 2024-06-01T02:17:11.839 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:11 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:11.540+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:35.068180+0000 front 2024-06-01T02:16:35.068257+0000 (oldest deadline 2024-06-01T02:16:59.768106+0000) 2024-06-01T02:17:12.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:11 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:11.844+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.463978+0000 front 2024-06-01T02:16:22.464056+0000 (oldest deadline 2024-06-01T02:16:44.764055+0000) 2024-06-01T02:17:12.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:11 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:11.844+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:27.065127+0000 front 2024-06-01T02:16:27.065136+0000 (oldest deadline 2024-06-01T02:16:52.965058+0000) 2024-06-01T02:17:12.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:11 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:11.844+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:38.866005+0000 front 2024-06-01T02:16:38.865958+0000 (oldest deadline 2024-06-01T02:17:01.765902+0000) 2024-06-01T02:17:12.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:11 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:11.844+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6828 osd.3 since back 2024-06-01T02:16:46.466989+0000 front 2024-06-01T02:16:46.466942+0000 (oldest deadline 2024-06-01T02:17:11.166902+0000) 2024-06-01T02:17:12.807 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:12 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:12.528+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:17:12.807 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:12 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:12.528+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:29.767566+0000 front 2024-06-01T02:16:29.767739+0000 (oldest deadline 2024-06-01T02:16:55.067675+0000) 2024-06-01T02:17:12.807 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:12 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:12.528+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:35.068180+0000 front 2024-06-01T02:16:35.068257+0000 (oldest deadline 2024-06-01T02:16:59.768106+0000) 2024-06-01T02:17:13.193 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:12 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:12.812+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.463978+0000 front 2024-06-01T02:16:22.464056+0000 (oldest deadline 2024-06-01T02:16:44.764055+0000) 2024-06-01T02:17:13.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:12 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:12.812+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:27.065127+0000 front 2024-06-01T02:16:27.065136+0000 (oldest deadline 2024-06-01T02:16:52.965058+0000) 2024-06-01T02:17:13.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:12 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:12.812+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:38.866005+0000 front 2024-06-01T02:16:38.865958+0000 (oldest deadline 2024-06-01T02:17:01.765902+0000) 2024-06-01T02:17:13.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:12 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6[120128]: 2024-06-01T02:17:12.812+0000 7f73354bd700 -1 osd.6 196 heartbeat_check: no reply from 172.21.15.116:6828 osd.3 since back 2024-06-01T02:16:46.466989+0000 front 2024-06-01T02:16:46.466942+0000 (oldest deadline 2024-06-01T02:17:11.166902+0000) 2024-06-01T02:17:13.609 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:13 smithi161 podman[150318]: 2024-06-01 02:17:13.342881446 +0000 UTC m=+5.444408147 container died 6b87d9299c8d26ff1466530771c14803eba2d44735d55e1437793c1380c1e2e6 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240528) 2024-06-01T02:17:13.609 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:13 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:13.505+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:17:13.609 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:13 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:13.505+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:29.767566+0000 front 2024-06-01T02:16:29.767739+0000 (oldest deadline 2024-06-01T02:16:55.067675+0000) 2024-06-01T02:17:13.609 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:13 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:13.505+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:35.068180+0000 front 2024-06-01T02:16:35.068257+0000 (oldest deadline 2024-06-01T02:16:59.768106+0000) 2024-06-01T02:17:13.609 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:13 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:13.505+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6828 osd.3 since back 2024-06-01T02:16:47.269869+0000 front 2024-06-01T02:16:47.269857+0000 (oldest deadline 2024-06-01T02:17:13.169844+0000) 2024-06-01T02:17:13.942 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:13 smithi161 podman[150318]: 2024-06-01 02:17:13.779304746 +0000 UTC m=+5.880831444 container cleanup 6b87d9299c8d26ff1466530771c14803eba2d44735d55e1437793c1380c1e2e6 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-06-01T02:17:13.942 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:13 smithi161 bash[150318]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6 2024-06-01T02:17:14.193 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:13 smithi161 podman[150333]: 2024-06-01 02:17:13.947291424 +0000 UTC m=+0.607009416 container remove 6b87d9299c8d26ff1466530771c14803eba2d44735d55e1437793c1380c1e2e6 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux ) 2024-06-01T02:17:14.693 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:14.535+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:17:14.693 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:14.535+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:29.767566+0000 front 2024-06-01T02:16:29.767739+0000 (oldest deadline 2024-06-01T02:16:55.067675+0000) 2024-06-01T02:17:14.694 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:14.535+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:35.068180+0000 front 2024-06-01T02:16:35.068257+0000 (oldest deadline 2024-06-01T02:16:59.768106+0000) 2024-06-01T02:17:14.694 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:14 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:14.535+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6828 osd.3 since back 2024-06-01T02:16:47.269869+0000 front 2024-06-01T02:16:47.269857+0000 (oldest deadline 2024-06-01T02:17:13.169844+0000) 2024-06-01T02:17:14.694 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:14 smithi161 podman[150424]: 2024-06-01 02:17:14.398109767 +0000 UTC m=+0.114325240 container create 8185d25b6c8e05f8d77040fabeaf23620f0e1e1690adc2befdfd61fa788961fd (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-01T02:17:14.694 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:14 smithi161 podman[150424]: 2024-06-01 02:17:14.300863867 +0000 UTC m=+0.017079369 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:17:14.694 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:14 smithi161 podman[150424]: 2024-06-01 02:17:14.617416153 +0000 UTC m=+0.333631636 container init 8185d25b6c8e05f8d77040fabeaf23620f0e1e1690adc2befdfd61fa788961fd (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-01T02:17:14.694 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:14 smithi161 podman[150424]: 2024-06-01 02:17:14.620762132 +0000 UTC m=+0.336977618 container start 8185d25b6c8e05f8d77040fabeaf23620f0e1e1690adc2befdfd61fa788961fd (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-06-01T02:17:15.193 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:14 smithi161 podman[150424]: 2024-06-01 02:17:14.714516953 +0000 UTC m=+0.430732459 container attach 8185d25b6c8e05f8d77040fabeaf23620f0e1e1690adc2befdfd61fa788961fd (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-01T02:17:15.194 INFO:journalctl@ceph.osd.6.smithi161.stdout:Jun 01 02:17:14 smithi161 podman[150424]: 2024-06-01 02:17:14.908995431 +0000 UTC m=+0.625210915 container died 8185d25b6c8e05f8d77040fabeaf23620f0e1e1690adc2befdfd61fa788961fd (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-06-01T02:17:15.519 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.6.service' 2024-06-01T02:17:15.558 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-01T02:17:15.558 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-06-01T02:17:15.559 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-06-01T02:17:15.559 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.7 2024-06-01T02:17:15.758 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:15 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:15.555+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:17:15.758 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:15 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:15.555+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:29.767566+0000 front 2024-06-01T02:16:29.767739+0000 (oldest deadline 2024-06-01T02:16:55.067675+0000) 2024-06-01T02:17:15.758 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:15 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:15.555+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:35.068180+0000 front 2024-06-01T02:16:35.068257+0000 (oldest deadline 2024-06-01T02:16:59.768106+0000) 2024-06-01T02:17:15.759 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:15 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:15.555+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6828 osd.3 since back 2024-06-01T02:16:47.269869+0000 front 2024-06-01T02:16:47.269857+0000 (oldest deadline 2024-06-01T02:17:13.169844+0000) 2024-06-01T02:17:15.759 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:15 smithi161 systemd[1]: Stopping Ceph osd.7 for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:17:16.193 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:15 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:15.915+0000 7ff8aca53700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-01T02:17:16.194 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:15 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:15.915+0000 7ff8aca53700 -1 osd.7 196 *** Got signal Terminated *** 2024-06-01T02:17:16.194 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:15 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:15.915+0000 7ff8aca53700 -1 osd.7 196 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-01T02:17:16.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:16 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:16.542+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:17:16.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:16 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:16.542+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:29.767566+0000 front 2024-06-01T02:16:29.767739+0000 (oldest deadline 2024-06-01T02:16:55.067675+0000) 2024-06-01T02:17:16.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:16 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:16.542+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:35.068180+0000 front 2024-06-01T02:16:35.068257+0000 (oldest deadline 2024-06-01T02:16:59.768106+0000) 2024-06-01T02:17:16.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:16 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:16.542+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6828 osd.3 since back 2024-06-01T02:16:47.269869+0000 front 2024-06-01T02:16:47.269857+0000 (oldest deadline 2024-06-01T02:17:13.169844+0000) 2024-06-01T02:17:17.943 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:17 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:17.561+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:17:17.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:17 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:17.561+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:29.767566+0000 front 2024-06-01T02:16:29.767739+0000 (oldest deadline 2024-06-01T02:16:55.067675+0000) 2024-06-01T02:17:17.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:17 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:17.561+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:35.068180+0000 front 2024-06-01T02:16:35.068257+0000 (oldest deadline 2024-06-01T02:16:59.768106+0000) 2024-06-01T02:17:17.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:17 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:17.561+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6828 osd.3 since back 2024-06-01T02:16:47.269869+0000 front 2024-06-01T02:16:47.269857+0000 (oldest deadline 2024-06-01T02:17:13.169844+0000) 2024-06-01T02:17:18.943 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:18.533+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:17:18.943 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:18.533+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:29.767566+0000 front 2024-06-01T02:16:29.767739+0000 (oldest deadline 2024-06-01T02:16:55.067675+0000) 2024-06-01T02:17:18.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:18.533+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:35.068180+0000 front 2024-06-01T02:16:35.068257+0000 (oldest deadline 2024-06-01T02:16:59.768106+0000) 2024-06-01T02:17:18.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:18 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:18.533+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6828 osd.3 since back 2024-06-01T02:16:47.269869+0000 front 2024-06-01T02:16:47.269857+0000 (oldest deadline 2024-06-01T02:17:13.169844+0000) 2024-06-01T02:17:19.943 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:19 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:19.546+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:17:19.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:19 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:19.546+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:29.767566+0000 front 2024-06-01T02:16:29.767739+0000 (oldest deadline 2024-06-01T02:16:55.067675+0000) 2024-06-01T02:17:19.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:19 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:19.546+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:35.068180+0000 front 2024-06-01T02:16:35.068257+0000 (oldest deadline 2024-06-01T02:16:59.768106+0000) 2024-06-01T02:17:19.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:19 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:19.546+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6828 osd.3 since back 2024-06-01T02:16:47.269869+0000 front 2024-06-01T02:16:47.269857+0000 (oldest deadline 2024-06-01T02:17:13.169844+0000) 2024-06-01T02:17:19.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:19 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:19.546+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.161:6804 osd.4 since back 2024-06-01T02:16:54.270740+0000 front 2024-06-01T02:16:54.270818+0000 (oldest deadline 2024-06-01T02:17:18.970963+0000) 2024-06-01T02:17:20.943 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:20 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:20.517+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6804 osd.0 since back 2024-06-01T02:16:22.365930+0000 front 2024-06-01T02:16:22.365762+0000 (oldest deadline 2024-06-01T02:16:43.366138+0000) 2024-06-01T02:17:20.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:20 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:20.517+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6812 osd.1 since back 2024-06-01T02:16:29.767566+0000 front 2024-06-01T02:16:29.767739+0000 (oldest deadline 2024-06-01T02:16:55.067675+0000) 2024-06-01T02:17:20.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:20 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:20.517+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6820 osd.2 since back 2024-06-01T02:16:35.068180+0000 front 2024-06-01T02:16:35.068257+0000 (oldest deadline 2024-06-01T02:16:59.768106+0000) 2024-06-01T02:17:20.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:20 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:20.517+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.116:6828 osd.3 since back 2024-06-01T02:16:47.269869+0000 front 2024-06-01T02:16:47.269857+0000 (oldest deadline 2024-06-01T02:17:13.169844+0000) 2024-06-01T02:17:20.944 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:20 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7[131270]: 2024-06-01T02:17:20.517+0000 7ff8a503f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.161:6804 osd.4 since back 2024-06-01T02:16:54.270740+0000 front 2024-06-01T02:16:54.270818+0000 (oldest deadline 2024-06-01T02:17:18.970963+0000) 2024-06-01T02:17:21.519 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:21 smithi161 podman[150530]: 2024-06-01 02:17:21.27176895 +0000 UTC m=+5.477415412 container died 400bb59c4249e6665af48d3d48b81ca9cc5a23e28c532b1b61a262fb30d3e7f4 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7, ceph=True, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-06-01T02:17:21.895 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:21 smithi161 podman[150530]: 2024-06-01 02:17:21.713724757 +0000 UTC m=+5.919371264 container cleanup 400bb59c4249e6665af48d3d48b81ca9cc5a23e28c532b1b61a262fb30d3e7f4 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-01T02:17:21.895 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:21 smithi161 bash[150530]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7 2024-06-01T02:17:22.193 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:21 smithi161 podman[150557]: 2024-06-01 02:17:21.899990588 +0000 UTC m=+0.619750038 container remove 400bb59c4249e6665af48d3d48b81ca9cc5a23e28c532b1b61a262fb30d3e7f4 (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, io.buildah.version=1.33.5) 2024-06-01T02:17:22.606 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:22 smithi161 podman[150645]: 2024-06-01 02:17:22.334429751 +0000 UTC m=+0.109966978 container create 2f72742c32bc9b7159b97e8b1f33516fccca6ea6af28d5638086130aee6d173d (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528) 2024-06-01T02:17:22.606 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:22 smithi161 podman[150645]: 2024-06-01 02:17:22.241663518 +0000 UTC m=+0.017200743 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-01T02:17:22.606 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:22 smithi161 podman[150645]: 2024-06-01 02:17:22.536631485 +0000 UTC m=+0.312168691 container init 2f72742c32bc9b7159b97e8b1f33516fccca6ea6af28d5638086130aee6d173d (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5) 2024-06-01T02:17:22.606 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:22 smithi161 podman[150645]: 2024-06-01 02:17:22.539803389 +0000 UTC m=+0.315340597 container start 2f72742c32bc9b7159b97e8b1f33516fccca6ea6af28d5638086130aee6d173d (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD) 2024-06-01T02:17:22.606 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:22 smithi161 podman[150645]: 2024-06-01 02:17:22.611484308 +0000 UTC m=+0.387021537 container attach 2f72742c32bc9b7159b97e8b1f33516fccca6ea6af28d5638086130aee6d173d (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-01T02:17:23.193 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jun 01 02:17:22 smithi161 podman[150645]: 2024-06-01 02:17:22.905921796 +0000 UTC m=+0.681459002 container died 2f72742c32bc9b7159b97e8b1f33516fccca6ea6af28d5638086130aee6d173d (image=quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2) 2024-06-01T02:17:23.930 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:17:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:17:23.709Z caller=refresh.go:98 level=error component="discovery manager notify" discovery=http msg="Unable to refresh target groups" err="Get \"http://172.21.15.161:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.161:8765: connect: connection refused" 2024-06-01T02:17:23.931 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:17:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:17:23.710Z caller=refresh.go:98 level=error component="discovery manager scrape" discovery=http msg="Unable to refresh target groups" err="Get \"http://172.21.15.161:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.161:8765: connect: connection refused" 2024-06-01T02:17:23.931 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:17:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:17:23.713Z caller=refresh.go:98 level=error component="discovery manager scrape" discovery=http msg="Unable to refresh target groups" err="Get \"http://172.21.15.161:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.161:8765: connect: connection refused" 2024-06-01T02:17:23.931 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:17:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:17:23.713Z caller=refresh.go:98 level=error component="discovery manager scrape" discovery=http msg="Unable to refresh target groups" err="Get \"http://172.21.15.161:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.161:8765: connect: connection refused" 2024-06-01T02:17:23.931 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:17:23 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:17:23.713Z caller=refresh.go:98 level=error component="discovery manager scrape" discovery=http msg="Unable to refresh target groups" err="Get \"http://172.21.15.161:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.161:8765: connect: connection refused" 2024-06-01T02:17:24.370 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@osd.7.service' 2024-06-01T02:17:24.409 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-01T02:17:24.409 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-06-01T02:17:24.410 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-06-01T02:17:24.410 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@prometheus.a 2024-06-01T02:17:24.543 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:17:24 smithi161 systemd[1]: Stopping Ceph prometheus.a for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:17:24.943 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:17:24 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:17:24.752Z caller=main.go:775 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-06-01T02:17:24.944 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:17:24 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:17:24.753Z caller=main.go:798 level=info msg="Stopping scrape discovery manager..." 2024-06-01T02:17:24.944 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:17:24 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:17:24.753Z caller=main.go:812 level=info msg="Stopping notify discovery manager..." 2024-06-01T02:17:24.944 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:17:24 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:17:24.753Z caller=main.go:834 level=info msg="Stopping scrape manager..." 2024-06-01T02:17:24.944 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:17:24 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:17:24.753Z caller=main.go:794 level=info msg="Scrape discovery manager stopped" 2024-06-01T02:17:24.944 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:17:24 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:17:24.753Z caller=main.go:808 level=info msg="Notify discovery manager stopped" 2024-06-01T02:17:24.944 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:17:24 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:17:24.753Z caller=manager.go:945 level=info component="rule manager" msg="Stopping rule manager..." 2024-06-01T02:17:24.944 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:17:24 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:17:24.753Z caller=main.go:828 level=info msg="Scrape manager stopped" 2024-06-01T02:17:24.944 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:17:24 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:17:24.753Z caller=manager.go:955 level=info component="rule manager" msg="Rule manager stopped" 2024-06-01T02:17:24.944 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:17:24 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:17:24.753Z caller=notifier.go:600 level=info component=notifier msg="Stopping notification manager..." 2024-06-01T02:17:24.944 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:17:24 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:17:24.753Z caller=main.go:1054 level=info msg="Notifier manager stopped" 2024-06-01T02:17:24.944 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:17:24 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a[62351]: ts=2024-06-01T02:17:24.753Z caller=main.go:1066 level=info msg="See you next time!" 2024-06-01T02:17:24.944 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:17:24 smithi161 podman[150878]: 2024-06-01 02:17:24.76300918 +0000 UTC m=+0.260934527 container died e977f6ca1d73ad506f7af13b6f248cf5e8a761c803c155b46dfce087f79bfa01 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-01T02:17:25.290 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:17:25 smithi161 podman[150878]: 2024-06-01 02:17:25.094120201 +0000 UTC m=+0.592045545 container cleanup e977f6ca1d73ad506f7af13b6f248cf5e8a761c803c155b46dfce087f79bfa01 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-01T02:17:25.290 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:17:25 smithi161 bash[150878]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a 2024-06-01T02:17:25.290 INFO:journalctl@ceph.prometheus.a.smithi161.stdout:Jun 01 02:17:25 smithi161 podman[150895]: 2024-06-01 02:17:25.261824461 +0000 UTC m=+0.496391702 container remove e977f6ca1d73ad506f7af13b6f248cf5e8a761c803c155b46dfce087f79bfa01 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-01T02:17:25.313 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@prometheus.a.service' 2024-06-01T02:17:25.349 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-01T02:17:25.349 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-06-01T02:17:25.350 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-01T02:17:27.292 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:17:27 smithi116 systemd[1]: Stopping Ceph node-exporter.a for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:17:27.292 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:17:27 smithi116 podman[166548]: 2024-06-01 02:17:27.138982 +0000 UTC m=+0.108503834 container died e347748d17bb33ab1f10ea6444ac5c3f24a297ee40e4451df4116a2bdd1c8603 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-06-01T02:17:27.723 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:17:27 smithi116 podman[166548]: 2024-06-01 02:17:27.536429115 +0000 UTC m=+0.505950994 container cleanup e347748d17bb33ab1f10ea6444ac5c3f24a297ee40e4451df4116a2bdd1c8603 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-06-01T02:17:27.723 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:17:27 smithi116 bash[166548]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a 2024-06-01T02:17:27.723 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:17:27 smithi116 bash[166575]: Error: no container with name or ID "ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter.a" found: no such container 2024-06-01T02:17:28.035 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:17:27 smithi116 systemd[1]: Stopping Ceph alertmanager.a for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:17:28.035 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:17:27 smithi116 podman[166560]: 2024-06-01 02:17:27.729219627 +0000 UTC m=+0.591016040 container remove e347748d17bb33ab1f10ea6444ac5c3f24a297ee40e4451df4116a2bdd1c8603 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-06-01T02:17:28.036 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:17:27 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-06-01T02:17:28.036 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:17:27 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-06-01T02:17:28.036 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:17:27 smithi116 systemd[1]: Stopped Ceph node-exporter.a for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:17:28.036 INFO:journalctl@ceph.node-exporter.a.smithi116.stdout:Jun 01 02:17:27 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@node-exporter.a.service: Consumed 4.863s CPU time. 2024-06-01T02:17:28.292 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:17:28 smithi116 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a[64300]: level=info ts=2024-06-01T02:17:28.040Z caller=main.go:557 msg="Received SIGTERM, exiting gracefully..." 2024-06-01T02:17:28.292 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:17:28 smithi116 podman[166625]: 2024-06-01 02:17:28.081663791 +0000 UTC m=+0.152490064 container died e0c297162dede4f605e15650dd5c8aee31163d9400299110e5d10a9ce1997313 (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-01T02:17:28.745 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:17:28 smithi116 podman[166625]: 2024-06-01 02:17:28.499475126 +0000 UTC m=+0.570301392 container cleanup e0c297162dede4f605e15650dd5c8aee31163d9400299110e5d10a9ce1997313 (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-01T02:17:28.745 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:17:28 smithi116 bash[166625]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a 2024-06-01T02:17:28.745 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:17:28 smithi116 bash[166651]: Error: no container with name or ID "ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager.a" found: no such container 2024-06-01T02:17:29.011 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:17:28 smithi116 podman[166640]: 2024-06-01 02:17:28.750971667 +0000 UTC m=+0.664018108 container remove e0c297162dede4f605e15650dd5c8aee31163d9400299110e5d10a9ce1997313 (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-01T02:17:29.011 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:17:28 smithi116 podman[166640]: 2024-06-01 02:17:28.826777171 +0000 UTC m=+0.739823614 volume remove ce7f7ac35efe9a0fb1e8ae759931fddc0c167ea28ca054f7e9da477ae1300ce2 2024-06-01T02:17:29.011 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:17:28 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-06-01T02:17:29.011 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:17:28 smithi116 systemd[1]: Stopped Ceph alertmanager.a for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:17:29.011 INFO:journalctl@ceph.alertmanager.a.smithi116.stdout:Jun 01 02:17:28 smithi116 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@alertmanager.a.service: Consumed 1.317s CPU time. 2024-06-01T02:18:03.320 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-01T02:18:05.119 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:18:05 smithi161 systemd[1]: Stopping Ceph node-exporter.b for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:18:05.443 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:18:05 smithi161 podman[152159]: 2024-06-01 02:18:05.124423988 +0000 UTC m=+0.108704199 container died 6e66182f5a1afcc0d2c6acbcdd5f7a12b8a739c4ebc2cca9d46319cf2ace23fa (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-06-01T02:18:05.927 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:18:05 smithi161 podman[152159]: 2024-06-01 02:18:05.538976853 +0000 UTC m=+0.523257064 container cleanup 6e66182f5a1afcc0d2c6acbcdd5f7a12b8a739c4ebc2cca9d46319cf2ace23fa (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-06-01T02:18:05.927 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:18:05 smithi161 bash[152159]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b 2024-06-01T02:18:05.927 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:18:05 smithi161 bash[152186]: Error: no container with name or ID "ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter.b" found: no such container 2024-06-01T02:18:05.927 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:18:05 smithi161 podman[152172]: 2024-06-01 02:18:05.739992107 +0000 UTC m=+0.616079383 container remove 6e66182f5a1afcc0d2c6acbcdd5f7a12b8a739c4ebc2cca9d46319cf2ace23fa (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-06-01T02:18:05.927 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:18:05 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-06-01T02:18:05.927 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:18:05 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-06-01T02:18:05.927 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:18:05 smithi161 systemd[1]: Stopped Ceph node-exporter.b for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:18:05.927 INFO:journalctl@ceph.node-exporter.b.smithi161.stdout:Jun 01 02:18:05 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@node-exporter.b.service: Consumed 4.882s CPU time. 2024-06-01T02:18:06.194 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:18:05 smithi161 systemd[1]: Stopping Ceph grafana.a for 393f964a-1fb9-11ef-bc9b-c7b262605968... 2024-06-01T02:18:06.194 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:18:06 smithi161 ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a[55888]: t=2024-06-01T02:18:06+0000 lvl=info msg="Shutdown started" logger=server reason="System signal: terminated" 2024-06-01T02:18:06.194 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:18:06 smithi161 podman[152240]: 2024-06-01 02:18:06.053618606 +0000 UTC m=+0.112786625 container died 00b0a78ee88e33518e46d676807c921bcf931285c1d4da94ee18cd04599e03ff (image=quay.io/ceph/ceph-grafana:8.3.5, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a, build-date=2022-03-28T10:36:18.413762, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, name=ubi8, vendor=Red Hat, Inc., version=8.5, architecture=x86_64, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.tags=base rhel8, io.openshift.expose-services=, description=Ceph Grafana Container, summary=Grafana Container configured for Ceph mgr/dashboard integration, release=236.1648460182, io.buildah.version=1.24.2, io.k8s.display-name=Red Hat Universal Base Image 8, distribution-scope=public, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-236.1648460182, com.redhat.component=ubi8-container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, maintainer=Paul Cuzner , vcs-type=git) 2024-06-01T02:18:06.564 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:18:06 smithi161 podman[152240]: 2024-06-01 02:18:06.376241054 +0000 UTC m=+0.435409062 container cleanup 00b0a78ee88e33518e46d676807c921bcf931285c1d4da94ee18cd04599e03ff (image=quay.io/ceph/ceph-grafana:8.3.5, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a, vendor=Red Hat, Inc., com.redhat.component=ubi8-container, version=8.5, release=236.1648460182, name=ubi8, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.tags=base rhel8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-236.1648460182, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.expose-services=, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, distribution-scope=public, vcs-type=git, io.k8s.display-name=Red Hat Universal Base Image 8, maintainer=Paul Cuzner , architecture=x86_64, description=Ceph Grafana Container, build-date=2022-03-28T10:36:18.413762, io.buildah.version=1.24.2) 2024-06-01T02:18:06.564 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:18:06 smithi161 bash[152240]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a 2024-06-01T02:18:06.564 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:18:06 smithi161 bash[152269]: Error: no container with name or ID "ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana.a" found: no such container 2024-06-01T02:18:06.943 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:18:06 smithi161 podman[152253]: 2024-06-01 02:18:06.569166861 +0000 UTC m=+0.516919528 container remove 00b0a78ee88e33518e46d676807c921bcf931285c1d4da94ee18cd04599e03ff (image=quay.io/ceph/ceph-grafana:8.3.5, name=ceph-393f964a-1fb9-11ef-bc9b-c7b262605968-grafana-a, io.buildah.version=1.24.2, io.openshift.expose-services=, io.openshift.tags=base rhel8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.k8s.display-name=Red Hat Universal Base Image 8, maintainer=Paul Cuzner , vendor=Red Hat, Inc., description=Ceph Grafana Container, distribution-scope=public, com.redhat.component=ubi8-container, summary=Grafana Container configured for Ceph mgr/dashboard integration, release=236.1648460182, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vcs-type=git, architecture=x86_64, build-date=2022-03-28T10:36:18.413762, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-236.1648460182, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, name=ubi8, version=8.5) 2024-06-01T02:18:06.943 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:18:06 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@grafana.a.service: Deactivated successfully. 2024-06-01T02:18:06.944 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:18:06 smithi161 systemd[1]: Stopped Ceph grafana.a for 393f964a-1fb9-11ef-bc9b-c7b262605968. 2024-06-01T02:18:06.944 INFO:journalctl@ceph.grafana.a.smithi161.stdout:Jun 01 02:18:06 smithi161 systemd[1]: ceph-393f964a-1fb9-11ef-bc9b-c7b262605968@grafana.a.service: Consumed 5.586s CPU time. 2024-06-01T02:18:29.706 DEBUG:teuthology.orchestra.run.smithi116:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-01T02:18:29.739 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-01T02:18:29.768 INFO:tasks.cephadm:Archiving crash dumps... 2024-06-01T02:18:29.770 DEBUG:teuthology.misc:Transferring archived files from smithi116:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-05-31_21:08:02-orch-squid-distro-default-smithi/7736361/remote/smithi116/crash 2024-06-01T02:18:29.771 DEBUG:teuthology.orchestra.run.smithi116:> sudo tar c -f - -C /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/crash -- . 2024-06-01T02:18:29.820 INFO:teuthology.orchestra.run.smithi116.stderr:tar: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-01T02:18:29.820 INFO:teuthology.orchestra.run.smithi116.stderr:tar: Error is not recoverable: exiting now 2024-06-01T02:18:29.822 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-05-31_21:08:02-orch-squid-distro-default-smithi/7736361/remote/smithi161/crash 2024-06-01T02:18:29.823 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/crash -- . 2024-06-01T02:18:29.904 INFO:teuthology.orchestra.run.smithi161.stderr:tar: /var/lib/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-01T02:18:29.904 INFO:teuthology.orchestra.run.smithi161.stderr:tar: Error is not recoverable: exiting now 2024-06-01T02:18:29.906 INFO:tasks.cephadm:Checking cluster log for badness... 2024-06-01T02:18:29.906 DEBUG:teuthology.orchestra.run.smithi116:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph.log | egrep CEPHADM_ | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-06-01T02:18:29.935 INFO:teuthology.orchestra.run.smithi116.stdout:2024-06-01T01:53:02.310647+0000 mon.a (mon.0) 374 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-06-01T02:18:29.935 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-06-01T02:18:29.935 DEBUG:teuthology.orchestra.run.smithi116:> sudo egrep '\[SEC\]' /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph.log | egrep CEPHADM_ | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-06-01T02:18:30.002 DEBUG:teuthology.orchestra.run.smithi116:> sudo egrep '\[ERR\]' /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph.log | egrep CEPHADM_ | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-06-01T02:18:30.071 DEBUG:teuthology.orchestra.run.smithi116:> sudo egrep '\[WRN\]' /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph.log | egrep CEPHADM_ | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-06-01T02:18:30.138 INFO:teuthology.orchestra.run.smithi116.stdout:2024-06-01T01:53:02.310647+0000 mon.a (mon.0) 374 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-06-01T02:18:30.139 INFO:tasks.cephadm:Compressing logs... 2024-06-01T02:18:30.139 DEBUG:teuthology.orchestra.run.smithi116:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-01T02:18:30.182 DEBUG:teuthology.orchestra.run.smithi161:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-01T02:18:30.204 INFO:teuthology.orchestra.run.smithi116.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-06-01T02:18:30.204 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph.log 2024-06-01T02:18:30.204 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-osd.2.log 2024-06-01T02:18:30.205 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-01T02:18:30.205 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph.log: /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-osd.3.log 2024-06-01T02:18:30.205 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-client.rgw.foo.smithi116.vzjrxj.log 2024-06-01T02:18:30.205 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph.audit.log: /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-osd.1.log 2024-06-01T02:18:30.207 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-client.rgw.foo.smithi116.vzjrxj.log: gzip -5 --verbose -- /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-mgr.y.log 2024-06-01T02:18:30.207 INFO:teuthology.orchestra.run.smithi116.stderr: 76.3% -- replaced with /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-client.rgw.foo.smithi116.vzjrxj.log.gz 2024-06-01T02:18:30.207 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-osd.0.log 2024-06-01T02:18:30.207 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-mon.c.log 2024-06-01T02:18:30.207 INFO:teuthology.orchestra.run.smithi161.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-06-01T02:18:30.208 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-client.rgw.smpl.smithi161.axjlzw.log 2024-06-01T02:18:30.208 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-client.rgw.foo.smithi161.ashqxz.log 2024-06-01T02:18:30.210 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-osd.0.log: 93.4% -- replaced with /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-01T02:18:30.210 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-mon.a.log 2024-06-01T02:18:30.211 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph.log 2024-06-01T02:18:30.212 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-client.rgw.smpl.smithi161.axjlzw.log: /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-client.rgw.foo.smithi161.ashqxz.log: gzip -5 --verbose -- /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-01T02:18:30.212 INFO:teuthology.orchestra.run.smithi161.stderr: 81.8% -- replaced with /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-client.rgw.smpl.smithi161.axjlzw.log.gz 2024-06-01T02:18:30.212 INFO:teuthology.orchestra.run.smithi161.stderr: 82.3% -- replaced with /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-client.rgw.foo.smithi161.ashqxz.log.gz 2024-06-01T02:18:30.212 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-osd.7.log 2024-06-01T02:18:30.212 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-osd.5.log 2024-06-01T02:18:30.212 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-osd.4.log 2024-06-01T02:18:30.213 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-osd.5.log: 88.3% -- replaced with /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-01T02:18:30.214 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-mgr.x.log 2024-06-01T02:18:30.226 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-client.rgw.smpl.smithi116.hokbmf.log 2024-06-01T02:18:30.227 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-osd.4.log: 92.9% -- replaced with /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-01T02:18:30.227 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-osd.6.log 2024-06-01T02:18:30.227 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-01T02:18:30.229 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-01T02:18:30.232 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph.cephadm.log: 91.0% -- replaced with /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-01T02:18:30.232 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-mon.b.log 2024-06-01T02:18:30.233 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-01T02:18:30.240 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-client.rgw.smpl.smithi116.hokbmf.log: gzip -5 --verbose -- /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/tcmu-runner.log 2024-06-01T02:18:30.241 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-01T02:18:30.245 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph.cephadm.log: 95.5% 76.5% -- replaced with /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-client.rgw.smpl.smithi116.hokbmf.log.gz 2024-06-01T02:18:30.246 INFO:teuthology.orchestra.run.smithi116.stderr: 94.7% -- replaced with /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-01T02:18:30.247 INFO:teuthology.orchestra.run.smithi116.stderr: -- replaced with /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-01T02:18:30.247 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-01T02:18:30.248 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/tcmu-runner.log: 87.3% -- replaced with /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/tcmu-runner.log.gz 2024-06-01T02:18:30.249 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-01T02:18:30.290 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-mon.b.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-01T02:18:30.319 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-volume.log: /var/log/ceph/cephadm.log: 90.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-01T02:18:30.324 INFO:teuthology.orchestra.run.smithi161.stderr: 89.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-01T02:18:30.342 INFO:teuthology.orchestra.run.smithi116.stderr: 92.2% -- replaced with /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-01T02:18:30.452 INFO:teuthology.orchestra.run.smithi116.stderr: 87.7% -- replaced with /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-mgr.y.log.gz 2024-06-01T02:18:30.977 INFO:teuthology.orchestra.run.smithi116.stderr: 92.7% -- replaced with /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-mon.c.log.gz 2024-06-01T02:18:31.011 INFO:teuthology.orchestra.run.smithi161.stderr: 92.8% -- replaced with /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-mon.b.log.gz 2024-06-01T02:18:31.174 INFO:teuthology.orchestra.run.smithi161.stderr: 88.7% -- replaced with /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-mgr.x.log.gz 2024-06-01T02:18:31.431 INFO:teuthology.orchestra.run.smithi116.stderr: 93.7% -- replaced with /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-osd.2.log.gz 2024-06-01T02:18:31.955 INFO:teuthology.orchestra.run.smithi161.stderr: 93.6% -- replaced with /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-osd.6.log.gz 2024-06-01T02:18:32.301 INFO:teuthology.orchestra.run.smithi161.stderr: 94.0% -- replaced with /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-osd.7.log.gz 2024-06-01T02:18:32.312 INFO:teuthology.orchestra.run.smithi116.stderr: 93.7% -- replaced with /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-osd.0.log.gz 2024-06-01T02:18:32.414 INFO:teuthology.orchestra.run.smithi116.stderr: 91.7% -- replaced with /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-mon.a.log.gz 2024-06-01T02:18:32.436 INFO:teuthology.orchestra.run.smithi161.stderr: 93.8% -- replaced with /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-osd.5.log.gz 2024-06-01T02:18:32.563 INFO:teuthology.orchestra.run.smithi116.stderr: 93.8% -- replaced with /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-osd.1.log.gz 2024-06-01T02:18:32.633 INFO:teuthology.orchestra.run.smithi161.stderr: 93.9% -- replaced with /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-osd.4.log.gz 2024-06-01T02:18:32.635 INFO:teuthology.orchestra.run.smithi161.stderr: 2024-06-01T02:18:32.635 INFO:teuthology.orchestra.run.smithi161.stderr:real 0m2.439s 2024-06-01T02:18:32.635 INFO:teuthology.orchestra.run.smithi161.stderr:user 0m9.715s 2024-06-01T02:18:32.635 INFO:teuthology.orchestra.run.smithi161.stderr:sys 0m0.409s 2024-06-01T02:18:32.677 INFO:teuthology.orchestra.run.smithi116.stderr: 93.8% -- replaced with /var/log/ceph/393f964a-1fb9-11ef-bc9b-c7b262605968/ceph-osd.3.log.gz 2024-06-01T02:18:32.679 INFO:teuthology.orchestra.run.smithi116.stderr: 2024-06-01T02:18:32.679 INFO:teuthology.orchestra.run.smithi116.stderr:real 0m2.486s 2024-06-01T02:18:32.679 INFO:teuthology.orchestra.run.smithi116.stderr:user 0m10.989s 2024-06-01T02:18:32.679 INFO:teuthology.orchestra.run.smithi116.stderr:sys 0m0.398s 2024-06-01T02:18:32.679 INFO:tasks.cephadm:Archiving logs... 2024-06-01T02:18:32.680 DEBUG:teuthology.misc:Transferring archived files from smithi116:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-31_21:08:02-orch-squid-distro-default-smithi/7736361/remote/smithi116/log 2024-06-01T02:18:32.680 DEBUG:teuthology.orchestra.run.smithi116:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-01T02:18:34.096 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-31_21:08:02-orch-squid-distro-default-smithi/7736361/remote/smithi161/log 2024-06-01T02:18:34.097 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-01T02:18:35.423 INFO:tasks.cephadm:Removing cluster... 2024-06-01T02:18:35.423 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 --force 2024-06-01T02:18:35.715 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 393f964a-1fb9-11ef-bc9b-c7b262605968 --force 2024-06-01T02:18:36.008 INFO:tasks.cephadm:Removing cephadm ... 2024-06-01T02:18:36.008 DEBUG:teuthology.orchestra.run.smithi116:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-01T02:18:36.024 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-01T02:18:36.039 INFO:tasks.cephadm:Teardown complete 2024-06-01T02:18:36.039 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-06-01T02:18:36.049 INFO:teuthology.task.clock:Checking final clock skew... 2024-06-01T02:18:36.049 DEBUG:teuthology.orchestra.run.smithi116:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-01T02:18:36.066 DEBUG:teuthology.orchestra.run.smithi161:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-01T02:18:36.078 INFO:teuthology.orchestra.run.smithi116.stderr:bash: line 1: ntpq: command not found 2024-06-01T02:18:36.082 INFO:teuthology.orchestra.run.smithi116.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-01T02:18:36.082 INFO:teuthology.orchestra.run.smithi116.stdout:=============================================================================== 2024-06-01T02:18:36.082 INFO:teuthology.orchestra.run.smithi116.stdout:^* hv01.front.sepia.ceph.com 3 6 377 44 +767us[ +797us] +/- 45ms 2024-06-01T02:18:36.083 INFO:teuthology.orchestra.run.smithi116.stdout:^+ hv02.front.sepia.ceph.com 3 8 377 108 -1144us[-1090us] +/- 73ms 2024-06-01T02:18:36.083 INFO:teuthology.orchestra.run.smithi116.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 43 +239us[ +239us] +/- 66ms 2024-06-01T02:18:36.083 INFO:teuthology.orchestra.run.smithi116.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-01T02:18:36.093 INFO:teuthology.orchestra.run.smithi161.stderr:bash: line 1: ntpq: command not found 2024-06-01T02:18:36.098 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-01T02:18:36.098 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2024-06-01T02:18:36.098 INFO:teuthology.orchestra.run.smithi161.stdout:^* hv01.front.sepia.ceph.com 3 6 377 42 +730us[ +758us] +/- 45ms 2024-06-01T02:18:36.098 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 40 -1230us[-1230us] +/- 74ms 2024-06-01T02:18:36.098 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 40 +178us[ +178us] +/- 66ms 2024-06-01T02:18:36.098 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-01T02:18:36.099 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-06-01T02:18:36.106 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-06-01T02:18:36.107 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-06-01T02:18:36.125 DEBUG:teuthology.orchestra.run.smithi116:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-01T02:18:36.228 DEBUG:teuthology.orchestra.run.smithi161:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-01T02:18:36.325 DEBUG:teuthology.orchestra.run.smithi116:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-06-01T02:18:36.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-01T02:18:36.356 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-06-01T02:18:36.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-01T02:18:36.386 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-06-01T02:18:36.396 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-06-01T02:18:36.502 INFO:teuthology.task.internal:Duration was 2005.513075 seconds 2024-06-01T02:18:36.503 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-06-01T02:18:36.541 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-06-01T02:18:36.541 DEBUG:teuthology.orchestra.run.smithi116:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-01T02:18:36.544 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-01T02:18:36.580 INFO:teuthology.orchestra.run.smithi116.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-01T02:18:36.581 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-01T02:18:37.076 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-06-01T02:18:37.076 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi116.front.sepia.ceph.com 2024-06-01T02:18:37.076 DEBUG:teuthology.orchestra.run.smithi116:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-01T02:18:37.138 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi161.front.sepia.ceph.com 2024-06-01T02:18:37.138 DEBUG:teuthology.orchestra.run.smithi161:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-01T02:18:37.167 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-06-01T02:18:37.167 DEBUG:teuthology.orchestra.run.smithi116:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-01T02:18:37.181 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-01T02:18:37.233 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-06-01T02:18:37.233 DEBUG:teuthology.orchestra.run.smithi116:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-01T02:18:37.248 DEBUG:teuthology.orchestra.run.smithi161:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-01T02:18:37.995 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-06-01T02:18:38.005 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-06-01T02:18:38.006 DEBUG:teuthology.orchestra.run.smithi116:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-01T02:18:38.030 DEBUG:teuthology.orchestra.run.smithi161:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-01T02:18:38.057 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-06-01T02:18:38.072 DEBUG:teuthology.orchestra.run.smithi116:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-01T02:18:38.074 DEBUG:teuthology.orchestra.run.smithi161:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-01T02:18:38.094 INFO:teuthology.orchestra.run.smithi116.stdout:kernel.core_pattern = core 2024-06-01T02:18:38.121 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = core 2024-06-01T02:18:38.142 DEBUG:teuthology.orchestra.run.smithi116:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-01T02:18:38.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-01T02:18:38.176 DEBUG:teuthology.orchestra.run.smithi161:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-01T02:18:38.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-01T02:18:38.197 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-06-01T02:18:38.206 INFO:teuthology.task.internal:Transferring archived files... 2024-06-01T02:18:38.207 DEBUG:teuthology.misc:Transferring archived files from smithi116:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-31_21:08:02-orch-squid-distro-default-smithi/7736361/remote/smithi116 2024-06-01T02:18:38.207 DEBUG:teuthology.orchestra.run.smithi116:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-01T02:18:38.274 DEBUG:teuthology.misc:Transferring archived files from smithi161:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-31_21:08:02-orch-squid-distro-default-smithi/7736361/remote/smithi161 2024-06-01T02:18:38.280 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-01T02:18:38.329 INFO:teuthology.task.internal:Removing archive directory... 2024-06-01T02:18:38.329 DEBUG:teuthology.orchestra.run.smithi116:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-01T02:18:38.332 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-01T02:18:38.374 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-06-01T02:18:38.385 INFO:teuthology.task.internal:Not uploading archives. 2024-06-01T02:18:38.386 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-06-01T02:18:38.403 INFO:teuthology.task.internal:Tidying up after the test... 2024-06-01T02:18:38.403 DEBUG:teuthology.orchestra.run.smithi116:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-01T02:18:38.406 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-01T02:18:38.418 INFO:teuthology.orchestra.run.smithi116.stdout: 265745 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 1 02:18 /home/ubuntu/cephtest 2024-06-01T02:18:38.429 INFO:teuthology.orchestra.run.smithi161.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 1 02:18 /home/ubuntu/cephtest 2024-06-01T02:18:38.431 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-06-01T02:18:38.443 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-06-01T02:18:38.703 INFO:teuthology.run:Summary data: description: orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} duration: 2005.5130751132965 failure_reason: '"2024-06-01T01:53:02.310647+0000 mon.a (mon.0) 374 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log' owner: scheduled_teuthology@teuthology success: false 2024-06-01T02:18:38.703 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-01T02:18:38.783 INFO:teuthology.run:FAIL